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ABSTRACT 


EQUALIZATION AND DETECTION FOR DIGITAL 
COMMUNICATION OVER NONLINEAR BANDLIMITED 
SATELLITE COMMUNICATION CHANNELS 

BY 

ALBERTO GUTIERREZ, JR. 


Doctor of Philosophy, Engineering 
Specialization in Electrical Engineering 
New Mexico State University 
Las Cruces, New Mexico, 1995 
Dr. William E. Ryan, Chair 



This dissertation evaluates receiver-based methods for mitigating the effects 
due to nonlinear bandlimited signal distortion present in high data rate satellite 
channels. The effects of the nonlinear bandlimited distortion is illustrated for 
digitally modulated signals. A lucid development of the low-pass Volterra discrete- 
time model for a nonlinear communication channel is presented. In addition, 
finite-state machine models are explicitly developed for a nonlinear bandlimited 

t 

satellite channel. 

A nonlinear fixed equalizer based on Volterra series has previously been stud- 
ied for compensation of noiseless signal distortion due to a nonlinear satellite chan- 


vi 



nel. This dissertation studies adaptive Volterra equalizers on a downlink-limited 
nonlinear bandlimited satellite channel. We employ as figure of merits perfor- 
mance in the mean-square error and probability of error senses. In addition, a 
receiver consisting of a fractionally-spaced equalizer (FSE) followed by a Volterra 
equalizer (FSE- Volterra) is found to give improvement beyond that gained by the 
Volterra equalizer. Significant probability of error performance improvement is 
found for multilevel modulation schemes. Also, it is found that probability of er- 
ror improvement is more significant for modulation schemes, constant amplitude 
and multilevel, which require higher signal to noise ratios (i.e., higher modulation 
orders) for reliable operation. 

The maximum likelihood sequence detection (MLSD) receiver for a nonlinear 
satellite channel, a bank of matched filters followed by a Viterbi detector, serves 
as a probability of error lower bound for the Volterra and FSE- Volterra equalizers. 
However, this receiver has not been evaluated for a specific satellite channel. In 
this work, an MLSD receiver is evaluated for a specific downlink-limited satellite 
channel. Because of the bank of matched filters, the MLSD receiver may be high in 
complexity. Consequently, the probability of error performance of a more practical 
suboptimal MLSD receiver, requiring only a single receive filter, is evaluated. 
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Chapter 1 

INTRODUCTION 


As the 20th century comes to a close, the field of telecommunications is 
coming of age with voice, video, and data communication systems operating over 
copper, cable, fiber, and wireless media. Today it is common for a person to turn 
on a television, tune to a cable channel, and get a weather report at any time 
of the day based on video obtained from weather satellites. University students, 
industry personnel, and computer enthusiasts needing information on practically 
any subject matter can connect to the Internet and download images, programs, 
and data for their research or enjoyment. The medical community commonly 
monitors patients and receives vital information remotely via medical telemetry. 
Law enforcement officers often have CB radios, pagers, computers connected to 
a central data base, and cellular telephones operating simultaneously from their 
mobile squad cars. Because of this recent “information explosion,” communication 
systems are being pushed to their capacity. In order to meet these demands, it 
is essential to design communication systems which make the most efficient use 
of the precious bandwidth resource. Meeting these demands coupled with the 
mature theory of modem communication systems makes it an exciting time to be 
working on almost any aspect of communication systems. 

! 

In particular, satellites provide unique capabilities not available from other 
forms of communication systems. First, is the capability of global coverage for 
commercial communications use such as in the INTELSAT satellites [1], remote 
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sensing as in EOS (earth observing system) satellites [2], and surveillance as in 
IDCSP (Initial Defense Satellite Communication Program) satellites [1], Second, 
satellites are capable of providing bandwidth second only to fiber. Currently, 
satellite systems are being designed to support data rates in the tens of gigabits 
per second. In order to provide sufficient fink margin, satellite channels employ a 
high power amplifier (HPA) often in the form of a traveling wave tube (TWT). 1 
However, the increasing demand for bandwidth and the desire to minimize satellite 
power consumption often means the TWT is driven at or near saturation. The 
end result is the introduction of nonlinear bandlimited signal distortion yielding 
nonlinear ISI (intersymbol interference). 

This dissertation evaluates receiver-based methods for mitigating the effects 
due to nonlinear bandlimited signal distortion. Specifically, Volterra equalizers, 
FSE-Volterra equalizers, maximum likelihood sequence detection (MLSD), and 
suboptimal MLSD receivers are evaluated. The results of this dissertation will 
serve as a baseline for the evaluation of more complex structures based on these. 
In addition, these results will help gauge the performance of hardware implemen- 
tations of these structures. 

The following section presents a simplified communication subsystem for a 

typical communications satellite. The communication subsystem is then reduced 

» 

1 Recently solid state amplifiers have become available which will likely replace the TWT as the 
HPA in new satellites. Since many existing satellites use TWT amplifiers, in this work TWT amplifiers 
will be considered. However, the equalization and detection methods discussed will be equally valid for 
solid state amplifiers. 
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to a model suitable for evaluating the performance of a satellite communication 
channel. Next, the effects of the nonlinearity and satellite bandlimiting filters on 
digitally modulated signals is illustrated. A literature review of existing compen- 
sation techniques for mitigating the effects of nonlinear bandlimited signal dis- 
tortion in various types of communication systems is then presented. The last 
section gives an overview of the chapters in this dissertation. 

1.1 Satellite Communication Background 

The idea of satellite communication systems was introduced by Arthur C. 
Clarke in his famous paper, published in 1945, entitled “Extra Terrestrial Relays” 
[3]. Although this paper was generally regarded as science fiction [4], within 25 
years Clarke’s ideas materialized into a mature technology. Satellites were first 
placed in orbit in the late 1950’s a few hundred kilometers above the earth. These 
satellites were known as LEO (low earth orbit) satellites and have continued to 
be used for remote sensing applications. However, GEO (geostationary equato- 
rial orbit) satellites have been preferred for commercial communications. Placing 
a GEO satellite in orbit (approximately 22,000 miles above the earth) has been 
preferred to the expensive tracking and control systems required for LEO satel- 
lites. However, by the 1990’s, the availability of powerful computing and signal 

f 

processing devices has made LEO satellites attractive for providing communica- 
tion services [4]. In this work we will not consider the special issues of tracking 
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and synchronization presented by LEO satellites, however the methods developed 
herein may very well be useful for the LEO scenario. 

1.1.1 A Satellite’s Communication Subsystem 

A simplified block diagram of a co mmuni cation and antenna subsystem for 
a typical “bent pipe” (transparent repeater) FDMA (frequency division multiple 
access) satellite is depicted in Fig. 1.1, [4, 5]. Each antenna may operate in both 
a transmit and receive mode. A diplexer separates the received antenna energy 
from the transmit energy. The received energy is routed to the satellite input filter 
which limits the uplink noise into the satellite. The output of the input filter then 
enters the receiver which consists of an LNA (low noise amplifier), a downconverter 
(e.g., converts from 6 to 4 gigahertz), and a receiver output filter which removes 
unwanted frequency components due to the downconversion operation. The signal 
then enters input multiplexer filters each -of which selects the frequencies entering 
a particular channel. The signal is then amplified by a TWT amplifier. 

After amplification, a transmit beam switch then selects the channels which 
will form the transmit beam of a particular antenna. The output multiplexer 
filters restrict the TWT output for each channel to the pre-assigned frequencies. 

The resulting signal is then sent to the output filter, diplexer, and antenna. The 

» 

output filter assures that the aggregate signal (including all the satellite channels 
for a particular antenna) lies within the preassigned satellite bandwidth. 
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Figure 1.1. Simplified communication subsystem for typical communications 
satellite. 

The bandwidth is typically divided into channels of 36 to 40 MHz, where each 
channel is handled by a different transponder. A transponder consists of the subset 
of the communications subsystem responsible for receiving and transmitting a 
single satellite channel for a bent pipe satellite. For example, a typical 6/4 satellite 
transponder is depicted in Fig. 1.2. The transponder receives the signal centered 
at 6 GHz from the receive antenna and subsequently downconverts it to 3.775 
GHz (approximately 4 GHz). The RF bandpass filters preceding and following 
the TWT amplifier represent the input and output multiplexer filters, respectively. 


1.1.2 A Satellite Communication System Model 

In order to determine the effectiveness of the various equalization and detec- 
tion methods, it is necessary to reduce the transponder model of Fig. 1.2 to one 
suitable for evaluation and analysis. In this work the focus is on the communi- 
cation system performance for an individual user so an individual transponder is 
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Local Oscillator 
2225 MHz 


Figure 1.2. Simplified 6/4 transponder, 
considered rather than the entire communication subsystem. Only the non-ideal 
effects due to the nonlinear bandlimiting are considered. Other affects such as ICI 
(inter-channel interference) are not considered. 

Fig. 1.3 represents a low-pass equivalent block diagram of a single-hop 
transponder communications link which accounts for the do min ant, performance- 
limiting components. A single-hop satellite link consists of an uplink from a 
transmitting earth station to the satellite and a downlink from the satellite to 
a receiving earth station. The figure contains transmitter, satellite channel, and 
receiver. Since the modulators, downconverters, diplexers, and demodulators nor- 
mally present at the transmitter, satellite, and receiver are assumed ideal, it is 
not necessary to account for them in the model. 

The data-bearing waveform J2n — nT), where T is the symbol interval, 
8(t) is the Dirac delta function [6], and dn is complex data, is filtered by the 
transmit filter hr(t). The satellite model consists of a pre-filter, hpr e (t), a TWT 

r 

high powered amplifier, and post-filter, hp St (t). As mentioned, the pre- and post- 
filters represent the input multiplexing and output multiplexing filters. Although 
in general the signal entering the satellite is subject to thermal noise, only the 
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Receiver 


Figure 1.3. Low-pass equivalent communication system model, 
noise at the receiver is assumed to be significant. Because the noise at the receiver 
is dominant, the system is referred to as a downlink-limited satellite system. The 
output of the receive filter is then sampled, equalized, and detected. Finally, the 
detector outputs estimates dn of the transmitted symbols d n . 

1.2 Nonlinear Distortion in Satellite Channels 

Before discussing the effects of the nonlinear bandlimited satellite channel on 
digitally modulated signals, a brief discussion of digital modulation formats for 
satellite communication systems is presented. The combined effect of the TWT 
nonlinearity and filtering is then illustrated for 8-PSK and 16-QAM systems. 
Next, a literature review of existing nonlinear distortion compensation methods is 
presented. The literature review distinguishes between ISI compensatioh methods 
based at the transmitter and receiver. 
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1.2.1 Digital Modulation Formats 


A typical modulation format for satellite communications is M-PSK. Because 
of power limitations and the nonlinear TWT amplifier, bandwidth efficient modu- 
lation formats, such as M-QAM, are not commonly employed for satellites. How- 
ever, 16-QAM has recently been considered for satellite communication [7]. Also, 
variations to the rectangular 16-QAM signal constellation have recently been con- 
sidered for satellite communication . In particular (4,12) with an inner circle of 4 
signal points and an outer circle of 12 signal points has been considered. In con- 
trast to 16-QAM the (4,12) signal points lie on concentric circles rather than on 
a square grid [8]. Other more sophisticated modulation methods, such as contin- 
uous phase frequency shift keying (CPFSK), will not be considered in this work. 
However, the equalization and detection techniques studied in this dissertation 
may also be useful for these modulation methods. 

1.2.2 Effects of Nonlinear Distortion 

The bandlimited nonlinearity in the transponder results from the pre-filter, 
TWT, post-filter combination. In this work the TWT is modeled, following Saleh 
[9], as a frequency-independent memoryless bandpass function. It is completely 
characterized by its AM/ AM and AM/PM conversions given by , 

>1 (r) = rxTTS, (AM/AM) (1.1) 

1 + PaT 

m = j-^3, (AM/PM) (1.2) 
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where r is the amplitude of the input waveform, and the parameters a a , /3 a , a <t>, 
and (3$ are obtained by a minimum mean-square error curve fitting procedure to 
experimental TWT data . If r(t) and 9(t) are the instantaneous input modulus 
and phase, respectively, of the TWT, then A [r(£)] and <b [r(£)] + 0(£) represent the 
instantaneous amplitude and phase, respectively, of the TWT output. For large 
r, the AM/ AM term becomes proportional to 1/r by the proportionality constant 
a a -//3a- Also, for large r the AM/PM term becomes the constant ct^/0^. 

An amplitude (magnitude, volts) and phase (radians) plot of the functions 
(1.1) and (1.2) with the parameters ct a = 1.9638, fi a = 0.9945, a# = 2.5293, (3 ^ = 
2.8168, is shown in Fig. 1.4. As is evident from the figure, the output amplitude 
given by (1.1) is normalized such that it is saturated at an input amplitude of 
unity. For small values of r (input magnitude, volts) the output magnitude (volts) 
and phase appear to be linear functions. However, as r approaches 1 the output 
voltage and phase begin to saturate. For r > 1 the output voltage begins to 
decrease and behaves as 1/r. Because the TWT is between two linear filters, the 
overall channel is a nonlinear system with memory. 

The effects of the nonlinear distortion have been studied extensively for digital 
radio links [10]. Fig. 1.5 illustrates an 8-PSK scatter plot of noiseless detector 
samples for the satellite channel of Fig. 1.3. In this case, the pre- and post-filters 
are 6th order butterworth with 3-dB bandwidths of 0.7bR s . The transmit filter 
is rectangular, hr(t) = 1, 0 < t < T, and the receive filter is matched to the 
transmitter. The TWT is driven at 0-dB input backoff. Here, input backoff is 
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Figure 1.4. TWT amplitude and phase plot, 
defined as in [11]. Referring to Fig. 1.4, an input backoff of X indicates that the 
average input signal power is decreased by X-dB with respect to the input signal 
power that causes saturation at the output. The scatter plot resembles an 8-PSK 
constellation with noise, however the clustering about the ideal signal point is due 
to linear and nonlinear ISI, not thermal noise. Also, the scatter plot resembles 
the effects of a bandlimited linear channel. However, as will be demonstrated in 
Chapter 2 the distortion is due to both linear and nonlinear components. 

Fig. 1.6 illustrates a 16-QAM scatter plot of noiseless detector samples for 
the channel model of Fig. 1.3. Other than the 16-QAM modulation an’d the fact 
that -the TWT is driven at 6-dB input backoff, the channel is identical to that for 
Fig. 1-5. It is evident that the inner constellation points are subject to different 
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Figure 1.5. 8-PSK Clustering. 

amounts of phase shift by the TWT than the outer points. Also the outer corner 
points receive less amplification by the TWT than the other outer signal points 
so that the outer constellation points appear to be on a circle. These effects are 
known as warping [10]. Thus, in addition to clustering, the 16-QAM constellation 
is subject to warping. 

1.2.3 Compensation Methods for Nonlinear ISI 

The predominant method of compensation for ISI in satellite channels is lin- 
ear adaptive equalization in the form of a tapped delay line filter. Several new 
methods of compensating for nonlinear distortion have been developed for digital 
radio, magnetic recording, telephony, as well as satellites. These methods can be 
separated into those which operate at the transmitter and at the receiver. 
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Figure 1.6. 16-QAM clustering and warping 
1.2. 3.1 Transmitter-Based Methods 

Transmitter-based methods for nonlinear digital radio systems include analog 
signal pre-distortion [10], data pre-distortion [12], and data pre-distortion with 
memory [13]. When these methods are made adaptive they require a feedback 
path from the output of the nonlinearity (at the transmitter) to the pre-distortion 
circuit. An adaptive data pre-distortion algorithm was first developed by Saleh 
and Salz [12], and later utilized by Karam and Sari for data pre-distortion with 
memory [13]. In this algorithm, the radial and phase error after the nonlinear- 
ity is measured and the pre-distorter is adjusted so as to decrease these errors. A 
pre-distorter (memoryless) can be implemented with a look-up table where the in- 
formation symbol serves as the address of the pre-distorted value. A pre-distorter 
with memory is implemented in much the same way except a concatenation of 
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past and present information symbols serve as the address to a memory which 
holds pre-distorted values. For systems with large modulation orders and many 
symbols of memory, the size of the predistortion memory device may become pro- 
hibitively large. However in [13], Karam and Sari have suggested methods for 
reducing the pre-distorter memory size. 

Another issue with pre-distortion with memory is that adaption may be slow 
sinCO for each memory location several cycles of adaption may be necessary. This is 
an issue because of the large memory size and adaption for each memory location 
depends on the frequency of occurrence of each symbol sequence. Despite the 
practical issues, pre-distortion with memory was found to give the best overall 
performance improvement for a nonlinear digital radio system when compared to 
other methods based both at the transmitter and receiver. Unfortunately, this 
method is not directly applicable to satellite systems since the adaption method 
requires a feedback path at the output of the nonlinearity. 

1.2. 3. 2 Receiver-Based Methods 

There are an abundance of compensation methods which are based at the 
receiver. These include the well known adaptive tapped delay line equalizer, 
decision feedback equalizer (DFE), fractionally-spaced equalizer (FSE), Volterra 
nonlinear equalizer, ISI cancellation, and MLSD. The performance of all of these 
methods for digital radio systems, with the exception of the FSE and MLSD, 
were studied by Karam and Sari [11]. A brief description of each of these methods 
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will be discussed below. In addition to these methods, neural network equalizers 
have generated some recent interest as nonlinear adaptive equalizers for magnetic 
recording [14], and satellite communication systems [15]. Because a neural network 
is very complex and is susceptible to convergence at local minima, the practicality 
of neural networks as adaptive equalizers is uncertain. 

Tapped Delay Line Equalizers - The symbol-spaced (synchronous) tapped 
delay line equalizer [16] is well known and is discussed in detail in Chapter 3. 
This device consists of a tapped delay line, with one tap per symbol, and the 
output is a linear combination of the taps. The tapped delay line equalizer is very 
effective in reducing the performance degradation due to linear ISI. However, it 
is not capable of eliminating nonlinear distortion even in the absence of noise. 
Also, the output spectrum of the symbol-spaced tapped delay line may be aliased 
due to symbol rate sampling. The FSE is similar to the symbol-spaced equalizer, 
however it has multiple taps per symbol. The FSE inputs are sampled values of 
the channel output, where the channel is sampled at greater than twice the highest 
frequency component (after demodulation). Thus, the FSE does not suffer from 
aliasing. Also, an adaptive FSE can compensate for sample timing offset and 
minimizes the mean-square error at the output of the equalizer by matching to 
the channel and reducing ISI. 

r 

Decision Feedback Equalizers - A DFE is an extension of the tapped delay 
line. In addition to the tapped delay line equalizer preceding the decision device, a 
DFE equalizer also includes a tapped delay line following the decision device. The 
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intention is to subtract ISI from the current symbol due to previously detected 
symbols. In the case of severe amplitude distortion, the DFE is very effective in 
removing ISI from previously detected symbols without enhancing the noise and 
offers a performance improvement (in the probability of error sense) compared to 
the tapped delay line equalizer [17, ch. 6]. This is because the previously detected 
symbols are no longer noisy. However, the DFE suffers from error propagation 
due to incorrectly detected symbols. 

ISI Cancellers - An ISI canceller is an extension to the DFE equalizer. The 
intention is to estimate and subtract from the current symbol ISI due to precursor 
symbols in addition to postcursor ISI. This is accomplished in two stages. First, 
preliminary decisions are made from which an estimate of precursor ISI is made. 
The precursor ISI estimate is then subtracted from an input to the final decision 
device. Second, as in the case of a DFE, postcursor ISI is estimated from final 
decisions and also subtracted from the input to the final decision device. This 
device is significantly more complicated than a DFE in that it requires an addi- 
tional decision device and several tapped delay lines. Wesolowski [18] has found 
that the cancellers do not always achieve a si gnifi cant improvement over DFE’s 
of similar complexity. 

Volterra Equalizers - All of the equalizer structures described thus far are 

f 

based on a linear combination of taps from a tapped delay line. These structures 
can be generalized to devices based on nonlinear combinations of taps from a 
tapped delay fine. These nonlinear devices are founded in the Volterra series 
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structure for a nonlinear communications channel. The modeling of nonlinear 
satellite links based on Volterra series was performed by Benedetto, Biglieri, and 
Daffara [19]. Benedetto and Biglieri [20] studied the performance of a Volterra 
series based nonlinear equalizer for a satellite channel. The equalizer was not 
adaptive and the performance was measured in improvement of signal to distortion 
ratio and did not account for noise. 

- MLSD Receivers - MLSD structures for nonlinear satellite channels were stud- 
ied by Mesiya, McLane, and Campbell [21] for binary sequences over a nonlinear 
satellite channel. Also, an MLSD receiver structure for nonlinear satellite chan- 
nels of higher order modulation formats (i.e., M > 2) was proposed by Benedetto, 
Biglieri, and Castellani [22]. However, the performance was not analyzed for a 
specific satellite channel. 

1.3 Overview of Chapters 

A satellite communication system is a bandpass system. However, for simula- 
tion and analysis it is efficient to model such a system as a low-pass discrete-time 
equivalent. The low-pass discrete-time equivalent model for a linear system with 
ISI is easily derived [17, 23]. The generalization of this model for a nonlinear 

bandpass system with memory is given by the low-pass discrete-time equivalent 

» 

Volterra series characterization [19]. It has also been suggested by several authors 
that a nonlinear satellite channel may be described as a finite-state machine [22, 
24]. Chapter 2 presents a lucid explanation of the low-pass discrete- time equiva- 
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lent model for a nonlinear bandlimited satellite channel. In addition, an explicit 
development of the finite-state machine (FSM) model is given including two spe- 
cial cases. First, a receiver with a single receive filter and detector is considered. 
Second, the receiver consists of a bank of matched filters and detector. For each 
of these special cases the FSM model yields a state table which may be used to 
analyze the performance of the nonlinear channel. 

•_ As previously discussed, a fixed Volterra equalizer, following the receive filter, 
for a noiseless satellite communication channel was introduced by Benedetto and 
Biglieri [20]. In addition, it has been suggested [22] that this structure may be 
adapted with the LMS (least mean-square) algorithm. However, the performance 
of this structure was not studied for a specific satellite channel. In Chapter 3, the 
probability of error and mean-square error performance of this structure is studied 
for various PSK and QAM modulation formats for a downlink limited nonlinear 
bandlimited satellite channel. When the receive filter is matched to the transmit- 
ter (as is typical in satellite systems) and the transmission bandwidth approaches 
the satellite bandwidth, then this configuration is no longer optimal in the sense 
of optimizing the signal to noise ratio. For this case, an adaptive FSE is useful in 
compromising between optimizing the signal to noise ratio and minimizing the ISI 

[25] . Chapter 3 demonstrates that an FSE followed by a Volterra equalizer gives 

» 

improved performance beyond that obtained from a Volterra equalizer. Also, it 
is found that a receive filter matched to the received pulse shape, ignoring the 
TWT, followed by a symbol spaced equalizer may replace the FSE with a small 


17 



loss in performance. In addition to evaluating the performance of Volterra and 
FSE-Volterra equalizers, Chapter 3 reviews the necessary background on symbol- 
spaced and fractionally spaced adaptive linear equalizers. 

Forney [23] has shown that the optimum receiver for a linear channel with 
ISI is a whitened matched filter followed by a nonlinear processor known as the 
Viterbi algorithm [26]. Benedetto et al. [22] has shown that the MLSD receiver for 
a nonlinear bandlimited satellite channel is a bank of matched filters followed by 
a Viterbi detector, however, the performance of this structure was not evaluated 
for a specific satellite channel. This MLSD receiver is optimum in the probabil- 
ity of error sense and serves as a lower bound to the Volterra and FSE-Volterra 
equalizers. In Chapter 4, the probability of error performance of the MLSD re- 
ceiver is studied for a specific down link limi ted satellite channel. Also, the rela- 
tionship between the matched filter bank outputs and the Viterbi algorithm path 
metrics is clearly delineated. Because of the matched filter bank, the MLSD re- 
ceiver may be high in complexity. Consequently the performance of a suboptimal, 
single receive filter receiver, is also studied. The finite-state machine models of a 
nonlinear communication channel are utilized for evaluating the performance of 
both MLSD receivers. In addition, background and justification for the MLSD 
receivers is presented. 

f 

A summary of the dissertation results is presented in Chapter 5. Also, the rel- 
ative performance of the Volterra, FSE-Volterra, MLSD, and suboptimal MLSD, 
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structures is discussed. Many variations to these receiver structures merit further 
study, these will be suggested in Chapter 5. 

This dissertation has been focused on the evaluation of receiver structures 
which effectively compensate for the nonlinear distortion caused by nonlinear ban- 
dlimited satellite channels. This endeavor has required a large effort in develop- 
ment of software tools for computer simulation of the various nonlinear effects 
aiid compensation methods. Many of the programs are software implementations 
of fundamental digital communications concepts. The more advanced programs, 
however, axe listed in Appendix A. 



Chapter 2 

COMMUNICATION SYSTEM MODELS 

A satellite communication system is a bandpass system. However, for simula- 
tion and analysis, it is efficient to model such a system as a low-pass discrete-time 
equivalent. The low-pass discrete-time equivalent model for a linear system with 
ISI is easily derived [17, 23]. The generalization of this model for a nonlinear 
bandpass system with memory is given by the low-pass discrete-time equivalent 
Volterra series characterization [19]. It has also been suggested by several authors 
that a nonlinear satellite channel may be described as a finite-state machine [22, 
24]. 

This chapter first reviews the low-pass discrete-time equivalent model for a 
linear communication system. Then, a lucid explanation of the low-pass equivalent 
model for a nonlinear bandlimited satellite channel is given. Finally, the finite- 
state machine (FSM) model for a no nlin ear bandlimited communication system 
is explicitly developed, including two special cases: one with a single receive filter 
and one whose receiver contains a bank of matched filters. For each of these cases, 
the FSM model yields a state table which may be used to analyze the performance 
of the nonlinear channel. 

! 

2.1 Low-Pass Discrete-Time Equivalent Model For a Linear System 

A low-pass equivalent co mmuni cation system is illustrated in Fig. 2.1. The 

transmit sequence dn is complex. The tr ansmi t signal s(t) has a pulse shape 
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Figure 2.1. Low-pass equivalent communication system, 
defined by hx(t) and is subsequently filtered by the channel h c (t). The input to 
the receiver r{t) is the sum of the channel output x(t) and an AWGN noise process 
w(t). The receive filter outputs the signal y(t) which is sampled at the symbol 
rate 1/T. The output of the detector is the estimate (decision) dn of the complex 
information symbol d n . 

As derived in [23], the discrete-time output of the receive filter is a weighted 
sum of past and present input symbols plus noise. The analog filters may be 
replaced with discrete-time filters which yield an identical set of inputs y n = y(nT ) 
to the detector and thus identical detector outputs. Also, the noise process may 
be moved to the detector input by an appropriate transformation. 

The input to the sampler may be written as . 

y(t) = Y^d k h(t- kT) + rj(t), (2.1) 

k 

where h(t) is the combined signaling waveform 

h(t) = hx(t) * hc(t) * (2.2) 

and * indicates convolution. The signal tj ( t) represents the noise at the 'output of 
the receive filter given by 

rj(t) = w(t) * h R (t). (2.3) 
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The sampled input to the detector is given by 

y n = y(nT ) - d k h n _ k + y n , (2.4) 

k 

where the noise samples, rj n , into the detector are given by r){nT ), and /i n _* = 
h[(n — k)T}. Equation (2.4) indicates that the input to the detector is a linear 
combination of past, present, and future discrete-time channel inputs with the 
addition of noise samples. 

* The noise samples at the output of the receive filter y n are in general colored. 
That is, in general the expectation E [77 n y n '] is nonzero for n =£ n . For the case 
of colored noise samples it is difficult to evaluate the performance of the given 
communication system. Consequently, it is desirable to whiten the noise samples 
via a noise whitening filter [23]. In this dissertation, the receive filter is either 
square-root raised cosine, or rectangular (i.e., h(t) = 1, 0 < t < T). Consequently, 
the noise samples are uncorrelated, so that a noise whitenin g filter is not required. 

The discrete-time equivalent channel model is described by equation (2.4). 
The summation over k is in general infini te, however for practical channels it is 
finite, so that h k « 0 for |fc| > L , where L is a positive integer. The equiva- 
lent discrete time model for the system of Fig. 2.1 is shown in Fig. 2.2, with 
the exception that the detector is not included in the figure. The input to the 
communication system is the finite sequence of information symbols {(4_ fc } for 

T 

\k\ < L. Each information symbol dn_ k is then multiplied by weight h k and input 
to a s ummin g device. The output of the channel model at discrete-time n is the 
sum of the weighted input symbols and noise samples rj n . 
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Figure 2.2. Discrete-time channel model for a linear system. 

2.2 Low-Pass Discrete-Time Equivalent Model For a Nonlinear System 

First, following [22], a continuous-time low-pass equivalent representation of 
a nonlinear downlink-limited transponder communications link is derived using a 
low-pass equivalent Volterra series. Next, as in the case of a linear system, the 
continuous-time low-pass equivalent representation is sampled to form the low- 
pass discrete-time model. 

2.2.1 Low- Pass Equivalent Volterra Series For a Bandlimited Nonlinear 
Channel 

The low-pass equivalent Volterra expansion for a general nonlinear channel 
with memory is given by [19, 22] 

00 roc roo 

y(t) = 53 / dn... dr 2 m + ik B (T 1 ,...,T 2 m + i) 

m=0 J -°° J ~°° 

m+1 2m+l 

ijx(t-Ti) Yl X*(t-7l ), (2.5) 

1=1 1=771 -f- 2 
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Figure 2.3. Low-pass equivalent nonlinear transponder communication system, 
where the input x (t) and output y (t) are low-pass equivalent signals, * denotes 
complex conjugate, and ks (r\, ...,r 2m +i) is the baseband equivalent Volterra ker- 
nel, as defined in [22, ch. 2]. The low-pass equivalent Volterra expansion will now 
be developed for the specific case of the low-pass equivalent nonlinear transponder 
model of Fig. 2.3. 

As indicated in the figure, the transmitted signal :r(£) is defined in terms of 
the complex data symbols d n . The baseband equivalent linear filter h up (t) rep- 
resents the cascade of linear filters preceding the nonlinearity g(-) (i.e., uplink 
filters), including the transmit filter and input multiplexing filter of the transpon- 
der. The nonlinear function g(-) represents the memoryless nonlinearity of the 
TWT amplifier, and will be defined in more detail below. The cascade of linear 
filters following the nonlinearity is represented by the linear filter hdwn{t) (i.e., 
downlink filters), including the transponder output multiplexing filter and receive 
filter. As in the case of the linear channel, in general, including the receive filter 
in the model causes the noise process j](t) to be colored. However, as indicated 
previously, the receive filters used in this dissertation are such that the noise sam- 
ples at the output of the receive filter are uncorrelated. The baseband equivalent 
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Figure 2.4. Memoryless quadrature nonlinearity, 
noise source preceding the detector, n(t) as in Fig. 2.3, is obtained by convolving 
the noise source preceding the receive filter with the impulse response of the re- 
ceive filter. The detector, which may include an equalizer, provides a decision on 
complex data symbol d n at discrete-time n. 

The AM/ AM and AM/PM conversions of the TWT amplifier is represented 
by the function g(-), which consists of the in-phase and quadrature functions &(•) 
and g q (-), given by [9] 

gi = P(r) cos [0(r)] , (2.6) 

g q = A(r) sin [<j>(r )] , (2.7) 

as shown in Fig. 2.4, [19, 9], where A(r) and <J)(r) are given by (1.1) and (1.2), 
respectively. The parameters 72 m+i are obtained from a Taylor series expansion 
of the nonlinearity g(-): 

gW) = £ 'w(J 2m+1 - (2-8) 

771=0 

The parameters 7 2m+ i consist of the in-phase and quadrature terms: 

72m+l = 7i,2m+l ~t~ (2*9) 


aO 



gj[a(0] + 7«,[a(0] 

♦ 
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where 7^2™+ 1, and, 7 9 ,2m+i, are the coefficients obtained from a Taylor series ex- 
pansion of gi(-) and g q (-), respectively. As is shown in [22], the presence of only 
odd orders in (2.8) is due to the bandpass nature of the communication system. 
Furthermore, for the low-pass equivalent communication system, application of 
(2.8) results in the following relationship between the input v(t) and output z(t) 
of the nonlinearity 

zm = f; 7wi w*)r +1 [»’«]” . (2.10) 

m =0 

The input to the nonlinearity v(t ) and output of the channel y(t) (including 
the effect of the receive filter) axe obtained by applying straightforward Unear 
system theory concepts: 

v(t) = f h up {r)x[t - r)dr, (2.11) 

J —OO 

and 

y(t)= f h dvm (T)z(t - r)dr. (2.12) 

J— OO 

Substituting (2.11) into (2.10) and then the result into (2.12) yields after simpU- 


fication 


00 TOO [OO [ OO 

y(t) = H72m+1 / dry... I dr 2m+1 dr • h dvm {T) 

0 J “°° 

m-hl 2m+l m-hl 

• n ^p(r r - r) n /i; p (r s - r) JJ x(t - r^) 

r=l r=m+2 i=l 

2m+l 

• n ^*(<-7i) 

f=m-h2 


(2.13) 


The low-pass equivalent Volterra kernels are obtained by comparing (2.13) to (2.5): 

/ oo m +1 2 m+l 

d^'hdwn{j }*\ J h~up(j~ r T ) (2.14) 

■oo - *t i O 


s=m-f2 
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Furthermore, accounting for the form of the input x(t) = ]£„ d n 8(t — nT), the 
Volterra series expansion of (2.5) simplifies to 

y(t) = ££••• £ k B (t-n 1 T,...,t-n 2m+ iT) 

m=Oni=—oo n 2m+i = — oo 

•d ni d n ^..A r ^ +l d nmJr2 ...d Tlim+l , (2-15) 

where d ni are the data inputs at discrete time n*. 

Expressions (2.14) and (2.15) define the low-pass continuous-time Volterra 
series for the transponder model of Fig. 2.3. Equation (2.15) expresses the time- 
domain transponder output as a nonlinear combination of past, present, and future 
information symbols d ni . Each nonlinear combination of input symbols is scaled 
by the respective Volterra kernel, k B ^ m +i (t — niT, — n 2m+ \T) . 

2.2.2 Discrete-Time Equivalent Model For a Bandlimited Nonlinear 
Transponder 

The low-pass equivalent discrete-time model for a nonlinear system is ob- 
tained by sampling (2.15) at time nT. Thus, the baseband equivalent discrete- 
time nonlinear channel model is described by 

OO OO OO 

yn = E E K B (n u ...,n 2m +i) 

m=0 ni=— oo n 2 m+i=-°° 

•dm dn 2 • • -dum+i dnjn+2 • ••d n2 m+ 1 > (2- 16) 

where K B (ni,..., n 2m +\ ) is the low-pass equivalent discrete-time 

f 

Volterra kernel given by 

K b (ni, ...,n 2m+ i) = k B (nT — n\T, ...,nT — n 2m+ iT) . (2.17) 
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Equation (2.16) represents the entire nonlinear channel including receive filter. 
In practice all channels have finite memory and nonlinearity of finite degree so 
that the summations in (2.16) are finite. 

2.3 Finite-State Machine Model 

A downlink-limited nonlinear co mmuni cations channel with finite memory 
ma^ be modeled with a finite-state machine (FSM) [22, ch. 10], [24]. In contrast to 
the low-pass discrete time Volterra characterization of a communications channel, 
the FSM model of a communications channel is easily obtained. However, it may 
require a large amount of storage. The model is useful in deriving the channel 
statistics as in [24] or for deriving a state table which may be employed directly by 
a Viterbi detector. The FSM model consists of a nonlinear transmitter in the form 
of a FSM and a receiver. The following section explicitly develops the nonlinear 
FSM transmitter. Then, two special cases of this model will be considered. The 
state tables obtained from both of these cases are employed in Chapter 4 for 
studying the performance of nonlinear bandlimited satellite channels. 

First, the discrete-time detector inputs will be derived for a receiver consisting 
of a single receive filter. A state table description of the channel is then obtained 
from the FSM model, where the state table contains a listing of each channel input, 

f 

state, and discrete-time detector input. Although for a nonlinear bandlimited 
satellite channel the single receive filter is suboptimal, it is the typical case. Thus, 
t his case is useful for analyzing the performance of a typical receiver. 


28 



Second, a receiver with a bank of matched filters is considered. As in the 
previous case, a state table description of the channel is obtained. However, in 
this case the state table listing contains the channel input, state, and oversampled 
outputs from the channel. If, as in the previous case, the state table were to 
contain the receive filter outputs for each input-state combination, the state table 
would become prohibitively large. Therefore, in this case the state table listing 
contains the oversampled outputs of the channel. The oversampled output of 
the channel is the output of the channel, over the time interval [nT, (n + 1)T), 
sampled at a rate greater than twice the highest frequency. In this case, the task 
of calculating the discrete-time filter outputs is left to the receiver. The method 
of obtaining the discrete time outputs for each filter in the filter bank is then 
discussed. A receiver consisting of a matched filter bank followed by a Viter bi 
detector is the optimal detector for this ch ann el. Therefore, this case is useful for 
analyzing the performance of the opt imum receiver. The relationship between the 
filter bank and the Viterbi algorithm path metrics will be discussed in Chapter 4. 

2.3.1 Nonlinear FSM Transmitter 

A FSM model useful for modeling a downlink-limited satellite channel is 
shown in Fig. 2.5. In this model, the transmitter and satellite channel (i.e., as in 

f 

Fig. 2.3) are modeled as a nonlinear transmitter in the form of a FSM. The input 
to the FSM is the discrete-time data sequence {a„} , € {0, 1, . . . , M — 1}, and 

the output are the chips h(t — nT, a n , cr n ), where each chip is zero outside the in- 
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Figure 2.5. FSM model of a communication system, 
terval [nT, (n + 1 )T), a n is the state, and M is the symbol set size. Therefore, the 
output. signal s(t) of the nonlinear transmitter is the sum of nonoverlapping chips: 

s(t) =J2h(t-nT,a n ,a n ). (2.18) 

n 

The input to the receiver is the signal s(t) plus noise w(t). The receiver filters, 
samples, and detects the signal to produce an estimate dn of the information 
symbol. 

Assuming that the communications system has a memory of L symbols, the 
state cr n of the FSM is the set of L previous channel inputs {a n -i, a n-2, — , 
Therefore, the nonlinear transmitter has M L unique states and M unique inputs. 
Consequently, the FSM can generate up to M L+1 distinct chips. 

The output signal of the nonlinear transmitter s(t) of Fig. 2.5 corresponds 
to the signal s(t ) of Fig. 2.6. Assume the system of Fig. 2.6 has a memory of 
L p past and Lf future symbols, thus a total memory of L — L p + Lj. .Then, the 
nth symbol interval at the output of the post-filter occurs over the time interval 
[nT + L p , (n + 1 )T + L p ). Therefore, the output s(t) of the FSM transmitter is 
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Figure 2.6. Low-pass nonlinear system for development of FSM models, 
written with respect to the signals from Fig. 2.6 as 

h(t - nT, an, a n ) = J ^g{^a k Kp{r - kT]^h pst {t - t)(It, (2.19) 
nT + Lp ^ t (n -4- 1)T -f- Lp 


2.3.2, Case I: Single Receive Filter 

The FSM model with a single receive filter is shown in Fig. 2.7. The output 
of the receive filter 7 (t) is sampled and detected to provide the estimated data 
symbol a n . The discrete time data inputs to the detector are expressed as 

7n = 7 (nT) = s n + T) n> (2.20) 

The noiseless detector inputs are given by 

s n = f h(r -nT, a*, a n )h R (nT -t)c!t, . (2.21) 

J — OO 

and the discrete-time noise samples by 

7] n = [ 1 u(r)h R (nT - r)dr. (2.22) 

7—00 

The noiseless discrete-time data inputs together with the input and state of 
the FSM may be listed in a state table. For example, suppose Fig. 2.7 models 
a QPSK satellite system with memory L = Lf + L P , where the memory consists 
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Figure 2.7. FSM model with a single receive filter, 
of one past symbol, Lf = 1, and one future symbol, L p = 1. A state table for 
this case contains M L+1 = 64 input-state combinations, and is fisted in table 2.1. 
The first column fists the input and the next two columns fist the state a n 
(i.e., the two previous inputs and a„_ i). The last column fists the noiseless 
complex discrete-time receive filter outputs s n (i.e., the noiseless detector inputs). 
The symbol a„_i is defined as the “punctual symbol,” since it determines in which 
quadrant the output resides. In the absence of distortion, the mapping from the 
punctual symbol to constellation point is defined in Table 2.2. 

Since the FSM model transmitter consists of a shift register, then the state 
transition statistics can be modeled by a Markov process. Also, the present- 
to-next state transitions are implicit. For example, with present state cr n = 
{an-i, On_ 2 } and input a„ the next state is given by cr n +i = {an, On-x}. 

This state table may be employed to derive the noiseless discrete time outputs 

» 

of a nonlinear transmitter. In such a case, the inputs are obtained randomly 

from the set {0, 1, . . . , M-l} . Noise samples r\ n may then be added to s n to form 

the input to the detector. The state table may then be used to derive the path 
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Table 2.1. State table for single receive filter receiver 


On 

O n S n 

0 

o 

B 

-0.7059 - 0.7059* 

1 

o 

B 

-0.7577 - 0.7059* 

2 

D 

B 

-.7059 - 0.7577* 

3 

Q 

B 

-0.7577 - 0.7577* 

0 

O 

B 

0.5178 - 0.7577* 

1 

D 

B 

0.4660 - 0.7079* 

2 

D 

B 

0.5178 - 0.7577* 

3 

D 

B 

0.4660 - 0.7577* 

| 

0 

B 

B 

0.7146 + 0.7146* 

1 

B 

B 

0.6628 + 0.7146* 

2 

B 

B 

0.7146 + 0.6628* 

2 

B 

B 

0.6628 + 0.6628* 


Table 2.2. Punctual symbol to constellation point mapping 
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Figure 2.8. FSM model with a matched-filter bank receiver, 
metrics for detection of the sequence of input symbols {an}. Alternatively, other 
information such as channel statistics or minimum distance may be derived from 
the state table. These topics will be discussed further in chapter 4. 

2.3.3 Case II: Filter Bank Receiver 

Fig 2.8 shows a communication system with a finite-state machine transmitter 
where the receiver consists of a bank of matched filters. The bank of matched fil- 
ters consists of a filter hi matched to each of the M L+1 waveforms h(t, On, cr n ) gen- 
erated by the transmitter. For example, the filter hi corresponding to h(t, a*, cr n ) 
has the response h*(T — t, On, a n ). 

As in the previous case, suppose the FSM model of Fig. 2.8 represents a 

QPSK satellite system with a memory of one past and one future symbol. As 

! 

before there are M L+l = 64 chips h(t, a^, a n ). Consequently, the filter bank has 

64 matched filters. If, as in the previous case, the state table were to contain 

the receive filter outputs for each input-state combination, the state table would 
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Table 2.3. State table for matched filter bank receiver 



0 

n 


h 2 

h 6 


0 

Q 

El 

-0.39 - 0.36? 

-0.37 - 0.38? 

-0.35 - 0.39? 

-0.37-0.38? 

l 

Q 

El 

-0.09 - 0.47? 

-0.37 - 0.38? 

-0.38 - 0.35% 

-0.37 - 0.38? 

2 

o 

El 

-0.49 + 0.04? 

-0.37 - 0.38* 

-0.31 - 0.41i 

-0.37- 0.38? 

3 

Q 

a 

-0.02 - 0.0? 

-0.37 - 0.38? 

-0.32 - 0.39? 

-0.37- 0.38? 



become prohibitively large (64 matched filters for the present example) . Therefore, 
in this^case the state table listing contains the oversampled outputs of the channel. 
For each input-state combination (a n ,a n ) , the oversampled output of the channel 
is the chip h(t,a n ,a n ) sampled at a rate greater than twice its highest frequency 
component. Table 2.3 contains a partial state table fisting for this case, where 
the sampling rate is four times the symbol rate. The punctual symbol is a n _i, 
and the punctual symbol to constellation point mapping is the same as for case 
I. Thus, the table lists only input state combinations corresponding to the third 
quadrant. The first column contains the input and the next two columns contain 
the state. The next four columns contain the four samples of the chip h(t, a n , a n ). 

As mentioned previously, a receiver consisting of a matched filter bank fol- 
lowed by a Viterbi detector is the optimal detector for this channel. In this case, 
however, it is necessary for the receiver to compute the discrete- time filter outputs 
during each si g n alin g interval. Given that the chip h(t,a n ,a n ) was transmitted 

f 

over the nth time interval, the output of the ?th filter is given by 

&n) = &n) T (2.23) 
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(2.24) 


where Si(a n ,a n ) is the noiseless output for the zth filter, given by 

fT 

Siicin, a n ) = / h(r, a„, (r n )hi(T - t)cIt. 

Jo 

The discrete-time noise samples for the zth filter are given by 

r(n+l)T 

Vi,n = / w(r - nT)hi(T - T)dr, (2.25) 

JnT 

where w(t) is the noise process at the input to the filter bank, as in Fig. 2.6. For 
the case of w(t) an AWGN process, then 77 ^ is a Gaussian random variable with 
variance 

•f-Y fMttfdt. (2.26) 

Since the state table contains the chip samples sampled at a rate greater than 
twice the highest frequency, the state table (e.g., table 2.3) contains the necessary 
information to optimally detect the input signal. Given the chip h(r, a*, cr n ) was 
transmitted, then the sampled input to the filter bank is given by the sequence 
{r 3 (a n , <r„)} , where j= 1, 2 ... N ss , and N ss is an integer representing the number 
of samples per signal. Because of practicality considerations, here we restrict N ss 
to an integer. The received samples are given by r J (on, a n ) = h 3 (a n) <r n ) + w 3 n , 
where h j (a n , a n ) is the jth sample of the chip h(t, On, <r n ), and is the jth sample 
of the noise process w(t) over the nth time interval. Thus, given that the chip 
h(t, On, a n ) was transmitted, the output of the zth filter is given by 

7 i(On, ^n) = <7n) + W 3 n ) (2.27) 

j=l 

where h{ is the jth sample of the filter hi(t). Note that the filter samples may 
be obtained from the chip samples contained in the state table. This equation 


may also be decomposed into a form corresponding to the signal and noise as in 
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(2.24) and (2.25), respectively. Therefore, given the sampled received sequence, 
the discrete-time output for each filter may be computed from the chip samples 
in the state table. 

2.4 Chapter Summary 

This chapter has reviewed the low-pass discrete-time equivalent model for 
a linear passband communication system. Next, beginning with the low-pass 
equivalent Volterra series, a lucid development of the low-pass Volterra discrete- 
time model for a nonlinear satellite communications channel was presented. The 
resulting model is a polynomial expression relating an output symbol at discrete 
time n to the past, present, and future, input symbols. Deriving this model for a 
particular channel is computationally intensive. 

The last section of this chapter presented a FSM model, where two special 
cases were considered. The discrete-time detector inputs were derived for the 
single receive filter receiver. Although this receiver is suboptimal, this case is 
useful for evaluating the performance of the typical receiver. In contrast to the 
low-pass discrete-time Volterra model, the FSM model is easily derived, however it 
may require a large amount of memory for storing the state machine. Despite the 
possibly large memory requirement, the model is more efficient, computationally, 

r 

than the Volterra model. In this case, the FSM model is simply a lookup table 
containing the discrete-time detector inputs whereas the Volterra model requires 
evaluation of a polynomial expression. 
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Finally, a state table listing appropriate for a receiver with a matched fil- 
ter bank was described. This state table listing contains samples of the chips 
h(t, On, a n ) sampled at a rate greater than twice the highest frequency compo- 
nent. Also, a method of computing the matched filter bank outputs from the 
state table entries was described. This state table may be used for evaluating the 
performance of the optimal receiver for the nonlinear bandlimited satellite chan- 
nel consisting of a matched filter bank followed by a Viterbi detector. As in the 
previous case, the state table is easily derived and the model is still more effi- 
cient than the Volterra model. Although in this case the filter bank outputs must 
be computed for each symbol interval, this additional computational requirement 
also exists for the Volterra model. 
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Chapter 3 

ADAPTIVE VOLTERRA EQUALIZERS FOR NONLINEAR 
SATELLITE CHANNELS 

A fixed Volterra equalizer for compensation of ISI due to a noiseless satellite 
communication channel has previously been studied [20]. In addition, it has been 
suggested [22] that this structure may be adapted with the LMS (least mean- 
square) algorithm. However, the performance of this structure has not been stud- 
ied for a specific satellite channel, as is done here. In addition, here a multiple-step 
size algorithm is used to improve the convergence characteristics of the equalizer. 

Since many of the concepts of linear adaptive filters apply to adaptive Volterra 
equalizers, first this chapter briefly reviews the related concepts from adaptive lin- 
ear filter theory. Next, the adaptive Volterra equalizer is introduced, and the con- 
ditions for convergence are analyzed. Then, the mean-square error and probability 
of error performance is studied for various PSK and QAM modulation formats. 

Because of the aliased frequency spectrum into the detector, due to symbol 
rate sampling, a symbol spaced adaptive equalizer can only act to modify this 
aliased (i.e., folded) spectrum, and is thus limited in its ability to compensate for 
ISI. In this case, an adaptive FSE followed by a Volterra equalizer is shown to give 

improved performance beyond that obtained from a T-spaced Volterra equalizer 

? 

following the receive filter. Also, the ability of an FSE to adaptively realize the 
optimum linear receive filter is reviewed. 
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w(t) 


Figure 3.1. Low-pass equivalent satellite communications channel. 

3.1 Linear Equalizer Background 

The linear equalizer is developed according to the linear communications 
system model shown in Fig. 3.1. For this discussion, the channel is represented 
by the linear filter h c (t). The input to the equalizer y n is the sampled output 
of the receive filter hn(t), and the equalizer output is z n . In the absence of 
a nonlinearity, the combined signaling pulse h(t) is obtained by a convolution 
h{t) = hT{t)*hc(t)*hji(t). The discrete-time equalizer inputs y n may be expressed 
as 

Vn ~ ^ ^ ~f~ T/ n , (3-1) 

k 

where hk = h(kT), and rj n = i i(nT) is the discrete-time noise process at the output 
of the receive filter. The noise process r)(t) is given by the convolution w(t)*hn(t). 

It has been shown that for every reasonable optimization criterion that the 
optimum receive filter is a matched filter followed by a tapped delay line [27]. The 
transversal equalizer (i.e., tapped delay line) is shown in Fig. 3.2. Its output z n 
is a linear combination of the input sequence {?/„} given by 

K 

Zn= Y2 Vn-kCk , (3.2) 

k=-K 

where the c* are complex weights. 
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Figure 3.2. Transversal filter. 

The optimum, in the MSE sense, tap settings for the infinite length transversal 
equalizer are found according to the inverse z-transform of [17, ch. 6] 

= IfizWiz- 1 ) + N 0 ’ (3 ' 3) 

where H (z) is the z-transform of the discrete time signaling pulse hk and Nq/2 
is the spectral density of the noise process w(t). If No is sufficiently small, then 
equation (3.3) indicates that C(z) approaches the inverse of the signaling pulse 
H(z). 

For the case of a finite length transversal filter, the optimum tap weights, 
Copt, are expressed by the matrix form of the Wiener-Hopf equation [28, ch. 5] 


= R -1 p, (3.4) 

where R is the (2K+1) X (2K+1) correlation matrix £[y n y*], p is the (2K+1) 
element cross-correlation vector £[yz*], and y n is the tapped delay line input 
vector at discrete-time n. 
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3.1.1 The LMS Algorithm 


As indicated by (3.3) and (3.4), the optimum MSE tap settings for the 
transversal equalizer are a function of the channel characteristics. Often these are 
not known a priori. An algorithm capable of adapting the equalizer coefficients 
such that the mean-square error is minimized without knowledge of the channel 
statistics is the LMS algorithm. This algorithm is simple yet very effective. 

The instantaneous error at the output of the equalizer is given by 

e n = dn (3-5) 

where dn is the desired response at time n. In the LMS algorithm, the update to 
weight c k at time n + 1 is obtained from the negative gradient of e n with respect 
to c k : 

c k {n + 1) = Cfc(n) - \nV Ck (e n e *) , (3.6) 

where fx is the step size and * denotes complex conjugation. Since the tap weight 
updates are obtained from the instantaneous square error (as opposed to MSE 
used in a true gradient search algorithm), the tap updates are noisy and result in 
an excess mean-square error beyond that obtained by the optimum tap weights 
given by (3.4). However, a small step size has the effect of averaging the updates. 
Conversely, the larger the step size the faster the equalizer convergence, however 
this results in a larger excess mean-square error. Evaluating (3.6) with e n given 
by (3.5) and Zn given by (3.2) yields the well known form of the LMS update 
equation [29, 31] 

c k {n + 1) = Cfc(n) + iieny'n-k- 
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(3.7) 



Equation (3.7) may be expressed in vector form 

c n +i = c n + //e n y*, (3.8) 

where c n is the tap weight vector at time n. 

The desired response d n in the expression for e n , equation (3.5), may be 
replaced by the receiver output d n if decisions are correct with moderate to high 
probability. Usually, this substitution can be made if the probability of error is 
le£s than 0.1. This mode of operation is known as decision-directed training and 
was invented by Lucky [32]. In this mode, the equalizer can track slow variations 
in the channel characteristics. 

3.1.2 MSE Performance of the LMS algorithm 

The MSE performance of the LMS algorithm is described as 

J = E[e n e n ] — J m in + J a i (3*9) 

where J min is the MSE obtained from the optimum tap settings, (3.4), and Ja is 
the excess MSE described above. The excess mean-square error Ja is difficult to 
evaluate mathematically, however the analysis can by simplified with certain as- 
sumptions [17, ch. 6]. First, it is assumed that the mean values of the equalizer 
coefficients c n have converged to their optimum values C op t . Second, the instanta- 
neous square error |e ra | 2 is assumed to be uncorrelated with the tap input vector. 2 
Under these assumptions, it is found that the excess mean-square error may be 

2 This assumption is not strictly true. However, it is noted by Proakis [17] that it simplifies the 


derivation and yields useful results. 
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expressed as 


K 


^ Y T 

i zr 


At 


(3.10) 


k=-K A ^) 2 

where A/t is the ktb. eigenvalue of the tap input correlation matrix. This expression 


explicitly shows the dependence of the mean-square error on the step size. 


3.1.3 Convergence of the LMS algorithm 

- Two types of convergence will be discussed: convergence in the mean and 
convergence of the excess mean-square error. Convergence in the mean refers to 
convergence of the mean tap vector E[c] to the optimum tap vector Copt as the 
number of iterations n approaches i nfini ty. By an appropriate change of basis [28, 
ch. 9], it is shown that the condition for convergence in the mean is for the step 
size to satisfy 

o (3.ii) 

^max 

where A max is the largest eigenvalue of the tap input correlation matrix. Also, 
in examining convergence in the mean it is found that the convergence rate (i.e., 
of E[ c] to Copt) is limited by the eigenvalue spread A max /A min , where A min is the 
smallest eigenvalue of the tap input correlation matrix. 

Unfortunately, expression (3.11) does not assure good mean-square error per- 
formance. Thus, it is useful to establish the conditions for which the excess mean- 
square error converges. Under the assumption that the sequence of tap input 
vectors y„ are independent (independence assumption) and that the error e n<opt 
(obtained by evaluating (3.5) with Cpp t ) is independent of all tap inputs, the fol- 
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lowing condition for convergence of the excess mean square error is obtained [33, 


ch. 8]. 3 


0 < p < 


(3,12) 


(2.ST + l)A max 

This expression indicates that for convergence of the excess MSE, the maximum 
step size is inversely proportional to the number of taps as well as A max . 


3.2 Adaptive Volterra Equalizer 

In this section, the structure of a Volterra equalizer is described. Next, the 
method of adaption is discussed including convergence and a multiple-step size 
adaption method. With methods similar to [35], the mean-square error and prob- 
ability of error performance of the adaptive equalizer is then evaluated via Monte- 
Carlo computer simulations. 

As discussed in chapter 2, the Volterra series characterization of a nonlinear 
communication channel provides a relationship between the discrete-time input 
symbols and the discrete-time channel output symbols. Recall, that equation 
(2.15) expresses the time-domain transponder output as a nonlinear combination 
of past, present, and future information symbols 

The Volterra series equalizer is motivated by the theory of the pth-order 
inverse for nonlinear Systems [20 - 37]. The pth-order inverse of a system 
H, consisting of a pth order Volterra series is defined as a system for which the 

3 In practice, results predicted by the independence assumption hold over a wide range of step values 


[33]. Other researchers have analyzed the LMS convergence without the independence assumption [34]. 

45 



Volterra series of the system Q, formed by the tandem connection of K~ l and H, 
is such that the 2nd through the pth-order Volterra operators of Q axe zero. Also, 
the pth-order pre-inverse is identical to the pth-order post-inverse. Thus, a pth 
order Volterra series with properly selected coefficients can remove the nonlinearity 
of a nonlinear system up to pth order. However, the theorem also states that 
the tandem connection of H and Q may produce nonlinear terms of higher-order 
(i.e., p+1, p+2, ...). These higher order terms, however, are negligible in weakly 
nonlinear systems. 

In light of the discussion on pth-order inverse, the discrete-time low-pass 
equivalent Volterra series characterization of a no nlin ear communications channel 
(2.16) suggests the form of the nonlinear Volterra equalizer. The output of the 
equalizer, z„, consists of a linear combination of all linear terms and all possible 
combinations of nonlinear terms of y n of odd degree and is given by [20] 

= J2c k y n - k + C *i M , fc3 Vn-k 1 Un-k 2 y n - k$ 

k k% &2 

-b X^^^XT^ C *i> fc 2> A; 3> fc 4.fc5yn-fcl2/n-k2l/n-teyn-fcj2/n-A:s + ' ' ' (3.13) 

fcl &2 ks &4 ks 

This expression consists of infinite summations of linear ISI terms and non- 
linear ISI terms of odd degrees. In practice, any channel has a finite memory and 
nonlinearity of finite degree, so that the summations in (3.13) are finite. 

Fig. 3.3 illustrates a block diagram of a 3-tap 3rd-order Volterra equalizer, 
where yo in the figure corresponds to the y n in (3.13). The samples from a tapped 
delay line are the inputs to a nonlinear combiner. The nonlinear combiner then 
outputs all single taps and all combinations of three taps. Each output from the 
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Figure 3.3. 3-tap 3rd-order Volterra equalizer, 
nonlinear combiner is then scaled by a weight Ck to form an input to the summing 
device. 

In [20], a significant reduction in complexity is proposed for M-PSK systems 
by eliminating terms from the nonlinear combiner of the form yiViVl = yj for 
i = k (and similarly for j = k), assuming y n is of modulus one. However, because 
y n contains a noise term, this reduced complexity equalizer suffers a performance 
degradation. This will be discussed further in the mean-square error performance 
section, Section 3.2.2. 

As indicated by equation (3.13), for each order m greater than one, the non- 
linear combinations consist of (m + l)/2 terms times (m — l)/2 terms which are 
conjugated. For example, for m — 3 the nonlinear combinations take the form 
y n —k\ Vn—k.2 Vri—k3 • This suggests that for m = 3 there are 18 unique 3rd order 
terms, N$. Therefore, the total number of terms in a no nlin ear Volterra equalizer 
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(a) (b) 



(c) (d) 


Figure 3.4. Scatter plots: (a) no equalization, (b) 7-tap linear, (c) 7-tap linear, 
3-tap 3rd-order, (d) 7-tap 3rd-order. 

is given by 

L = N+f2 N m, (3.14) 

m odd 

where ip is the highest nonlinear term in the equalizer, and N m is the number of 
terms of order to. For the Volterra equalizer of Fig. 3.3, the total number of terms 
is 21. It is also possible to have a nonlinear equalizer with N-tap linear compen- 
sation, but with a subset, Ns, of the N taps for ming nonlinear combinations. 

The ability of the Volterra equalizer to mitigate the effects of nonlinear ISI 
are illustrated in the' 8-PSK scatter plots of Fig. 3.4. For this example, the 
transmit filter has a rectangular pulse shape and the receive filter is matched to 
the transmit pulse shape. The satellite pre- and post-filters are 6th order low-pass 
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butterworth with a 3 dB cutoff frequency of 0.75 R s . The noise signal at the input 
to the receiver is set to zero (i.e., w(t) = 0) and the TWT is driven at 0 dB backoff. 

For Fig. 3.4, the Volterra equalizers contained redundant terms and the total 
number of terms is given by 

L' = £ 1\T. (3.15) 

m=l 
m odd 

In this case, the number of terms for each nonlinear order is N m where all possible 
terms of a particular order are included including redundant terms. This is the 
case for all of the Volterra equalizers evaluated via computer simulation in Sections 
3.2.3 and 3.2.4. This will affect the rate of convergence since in effect the step size 
is increased for the redundant terms. However, the converged mean-square error 
and probability of error results are not affected. Including the redundant terms 
simplified the software implementation of the Volterra equalizer. 

Fig. 3.4 (a) shows a scatter plot of the receive filter output without equaliza- 
tion. In Fig. 3.4 (b), the receiver output is equalized with a 7-tap linear equalizer. 
Figs. 3.4 (c) and (d) illustrate the equalizer output of a 7-tap linear, 3-tap 3rd- 
order equalizer (7 linear and 27 3rd-order terms) and a 7-tap 3rd-order equalizer 
(7 linear and 343 3rd-order terms), respectively. As is evident in Fig. 3.4, the 
Volterra equalizers reduce the signal distortion beyond that of the linear equalizer. 
Also, the 7-tap 3rd-order equalizer reduces the nonlinear distortion beyond that 
of the 7-tap linear, 3-tap 3rd-order device. In the following sections, 3.2.3 and 
3.2.4, the performance of these devices will be evaluated in the presence of noise. 
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3.2.1 Volterra Equalizer Adaption. 


In the following discussion, the method of adaption and associated conver- 
gence for the nonlinear Volterra equalizer are discussed. First, the form of the 
MSE at the output of the equalizer is described. Then, the LMS weight update 
algorithm is described in terms of the nonlinear Volterra equalizer. The conver- 
gence of the equalizer is then studied and compared to the linear equalizer. The 
nonlinear Volterra equalizer convergence is found to be slower than that of a lin- 
ear equalizer with the same number of taps. Consequently, a multiple-step size 
LMS algorithm which improves the convergence characteristics is described. 

3. 2. 1.1 MSE Output of the Nonlinear Volterra Equalizer 

Denote the output of the nonlinear combiner at time n by the complex vector 
u n = [uo(n) tq (n) ••• u/,_i(rc)] T , where T denotes transpose, iq(n) is the ith 
nonlinear combiner output at time n, L is given by (3.14), and each u, (i = 
0, 1, ..., L — 1) is an output of the no nlin ear combiner. Denote the complex weight 
vector at time n by c n = [c 0 (n) c x (n) • • • cx,_i(n)] r , where Cj(n), is the weight 
multiplying Ui(n). Then, the output Zn of the Volterra equalizer at time n is given 
by 

2n = <£u n , (3.16) 

Define the nonlinear combiner output correlation matrix as 

R = E[u„u^], (3.17) 
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where £[ 2 ] denotes the expectation of x, and H denotes Hermitian transpose. As 
for the linear case, we define the error e n at the output of the equalizer as 

fin = d n z n , (3.18) 

where d n is the desired response at time n. The cost function or MSE at time n 
is defined as 

J n = E[e„e*j (3.19) 

By substituting (3.18) and into (3.19) and with straightforward manipulations the 
following expression for the MSE is obtained 

J n =cr 2 d - c£p - p*< + c^Rc*, (3.20) 

where is the variance of the desired response, p = E[u n d*] is the correlation 
of the tap input vector and desired response, and * denotes complex conjugate. 
Thus, the MSE varies with time, and is a quadratic function of the tap weights as 
it was for the linear equalizer. Thus, the LMS algorithm may be used for adaption 
as we now discuss. 

3. 2. 1.2 The LMS Algorithm in Relation to the Nonlinear Volterra 
Equalizer 

Since the nonlinear combining occurs before the tap weights, the outputs of 
the no nlin ear combiner may be considered inputs to a linear adaptive filter. Also, 
since the MSE is a quadratic function of the tap weights, the LMS algorithm may 
be employed for adaption. The weight update equation is given by 

C n +1 = C n + fj,e n u£. (3.21) 
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Thus, the weight update equation is the same as for the linear equalizer. However, 
in the case of the nonlinear Volterra equalizer weights for nonlinear terms are 
included. 

3. 2. 1.3 Convergence of the Nonlinear Volterra Equalizer 

Define the correlation matrix of the inputs to the N-tap delay line whose 

outputs are inputs to the nonlinear combiner as (e.g., as in Fig. 3.3) 

R = E[ [y„jf ], (3.22) 

where y n is the tap input vector at time n. Denote the eigenvalues of R as Aj and 

the eigenvalues of R as ctj. From [28, ch. 4] we have the following two expressions 

for the maximum and minimum eigenvalues of R. 

x H fix 

Otmax = max — rr — : x € C , i^O (3.23) 

x n x 

and 

(*min = min — tv — : x € C^, x ^ 0 (3.24) 

where C L is complex vector space of dim ension L. 

Equation (3.23) states that the largest eigenvalue of R is obtained by the 
largest amount by which any vector is amplified by vector multiplication. Since 
from (3.14) L > N, R is a submatrix of R , and C C L , we may conclude 
that ewe > A m ax and a min < Amin. Furthermore, considering (3.12) with 2 K + 1 
replaced by L or N , we have that 

Mm ax < Mmaxj (3.25) 
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where fimax is the maximum allowable step size for the Volterra equalizer and 
Mmax is the maximum step size for the corresponding linear equalizer. Note that 
£l max is smaller than /z max by at least the proportion of weights in the nonlinear 
equalizer (L) to the number of linear weights (N). In addition to the maximum 
step size being smaller for the nonlinear Volterra equalizer, the eigenvalue spread is 
potentially greater for the nonlinear Volterra equalizer as indicated above. Thus, 
the convergence rate is likely to be much slower for the Volterra equalizer. 

3. 2. 1.4 Multiple-Step Size Algorithm 

In the next section, the sensitivity to the step size is illustrated with an 
example. However, before considering such an example, it is useful to investigate 
a method for improving the convergence characteristics of the nonlinear equalizer. 
This can be done with the multiple-step size LMS algorithm [38-41], With this 
algorithm each weight may be updated with a unique step size. This leads to the 
update equation 

c n +i = c n + Mce n u*. (3.26) 

where Md is a diagonal matrix with the step sizes m (corresponding to weight 
Cj(n)) along the main diagonal and zeros elsewhere. With the nonlinear equalizer, 
it has been found useful to choose a larger step size for linear terms and smaller 
step sizes for the nonlinear terms. 

As shown in [38], for any positive choice of Me, the mean-square error will 
converge if the values of Mp are sufficiently small. However, if all the diagonal 
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Figure 3.5. 3- tap reduced and non-reduced MSE performance for a QPSK sys- 
tem. 

elements of M# are equal then the conditions for convergence are given by (3.11) 
and (3.12) for convergence of the mean tap vector and excess mean-square error, 
respectively. Alternatively, as in the linear filtering case [41], the update constant 
may be replaced with n R so that all the eigenvalues become unity and all modes 
decay at the same rate. 

3.2.2 MSE Performance 

In this section, the MSE performance results obtained from computer simu- 
lation are presented. The parameters for the systems simulated are the same as 
that for Fig. 3.4. 

Fig. 3.5 plots the mean-square error versus adaption time in symbols for 
various 3-tap equalizers for a QPSK system. The equalizers are a 3- tap linear, 3- 
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tap 3rd-order, and 3-tap 3rd-order reduced complexity. The reduced complexity 
equalizer has the tap weights of the terms yiyjy). set to zero for i = k or j = k, as 
discussed previously. In all cases y =1CT 3 . The MSE estimate is obtained by ap- 
propriately scaling a 512-symbol running sum of the squared error at the output 
of the equalizer. This method gives a quick but biased estimate of the MSE espe- 
cially before the equalizer has converged. After the equalizer has converged, the 
method gives a good estimate of the MSE. Although ensemble averaging of learn- 
ing curves is the proper method for estimating MSE, the computer time necessary 
for such an approach was prohibitive. The figure illustrates that for Eb/No = 10 
dB the reduced equalizer has performance (in MSE) comparable to that of the 
linear equalizer. As the Eb/No increases, the reduced equalizer performance ap- 
proaches that of the non-reduced equalizer but still has a performance loss even 
at very high Eb/No. The figure also illustrates that the convergence time for the 
3rd-order equalizer is on the order of 100,000 symbols, although most of the MSE 
reduction is achieved in about 10,000 symbols. 4 

Fig. 3.6 illustrates the advantage of multiple-step size adaption for a QPSK 
system with an Eb/No of 9 dB. The curve illustrating large jumps and higher 
overall mean-square error is for a 3-tap 5th-order equalizer with a single step size 
of 10 -3 . The lower curve is for a 3-tap 5th-order equalizer with multiple-step size 
adaption and step sizes of 10 -3 , 10~ 4 , and 10~ 5 , for the linear, 3rd-order, and 5th- 
order tefins, respectively. The single step size equalizer is evidently unstable at 

4 As noted previously, these equalizers contain redundant terms, where the total number of terms 
is given by (3.15). The redundant terms are expected to affect the convergence rate. 
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Figure 3.6. Multiple-step size adaption for a QPSK system, 
this adaption rate. In order to stabilize the single step size equalizer, the step size 
may be reduced but at the expense of a much slower convergence time. The figure 
illustrates that the multiple-step size algorithm lessens the penalty of a slower 
convergence rate while improving performance in MSE. 

Fig. 3.7 illustrates .the noiseless MSE performance of the 7-tap linear, 7-tap 
3rd-order, and 7-tap linear with 3-tap 3rd-order equalizers. As is clear from Fig. 
3.4, the 7-tap 3rd-order equalizer gives the best MSE performance with greater 
than '5 dB improvement compared to the linear equalizer. The 7-tap linear with 
3-tap 3rd-order equalizer obtains approximately 1.5 dB improvement compared 
to the linear equalizer. 
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Figure 3.7. Noiseless MSE performance for a QPSK system. 

3.2.3 Probability of Error Performance 

The probability of bit error (Pb) performance for QPSK and 8-PSK systems 
is shown in Fig. 3.8. The system parameters are the same as for Fig. 3.4. For the 
QPSK system, the equalizers are a 5-tap, linear, and 5-tap 3rd order. For the 8- 
PSK system the equalizers are a 7-tap linear, 7-tap 3rd-order, and 7-tap linear with 
3-tap 3rd-order. Although according to Figs. 3.5 and 3.7, the MSE performance 
for a 3rd-order equalizer is significantly better than for a linear device, there is no 
improvement in Pb for QPSK. 

Some insight may . be obtained by looking at a scatter plot of the equalizer 
output, Fig. 3.9. For this case it has been determined that the signal to distortion 
ratio Is approximately 15 dB whereas the symbol energy to noise spectral density 
ratio is 13 dB (i.e., Eb/No =10 dB). With the distortion below or at a comparable 
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Figure 3.8. QPSK and 8-PSK probability of error performance, 
level to the noise, the equalizer can significantly reduce the mean-square error by 
reducing the component of the noise in the radial direction, however, this noise 
reduction is orthogonal to the direction which leads to a decision error. Thus, 
although the mean-square error is significantly reduced, the probability of error is 
not improved. This result is unique to M-PSK when the distortion is comparable 
to or below the noise level. 

In the 8-PSK case, the noise level is below the distortion level at Pb =10 -5 
(see Fig. 3.8) and the 3rd-order nonlinear equalizers give a 0.5 dB improvement in 
Pb compared to the linear equalizer. The 7-tap 3rd-order equalizer (343 3rd-order 
terms) shows a slightly degraded performance compared to the linear equalizer at 
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Figure 3.9. Scatter plot of 7-tap 3rd-order equalizer output, 
low Eb/No due to noise enhancement at the nonlinear equalizer output. The 7-tap 
linear, 3-tap 3rd-order equalizer (27 3rd-order terms) gives the same performance 
improvement at high Eb/No at a much lower complexity compared to the former. 

Thus far only constant modulus systems have been considered. Fig. 3.10 

illustrates the performance in P b for a 16-QAM system. For this case the satellite 

parameters are the same as for Fig. 3.4, however the transmitter and receiver 

differ. The transmitter has a square root raised cosine pulse shape with 100% 

excess bandwidth and the receiver is matched to the transmitter. In the case of 

PSK, in order to minimize the nonlinear distortion, it is desirable to transmit 

constant modulus signals, so it is typical to transmit with a rectangular pulse 

shape. However, since QAM employs multi-level signals, they are not constant 

modulus. Since the modulation is not constant modulus, a square root raised 
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Figure 3.10. 16-QAM probability of bit error performance, 
cosine pulse shape is employed at the transmitter and receiver as is usual for this 
scheme. 

Several equalizer configurations are considered: 9-tap linear, 9-tap 3rd-order 
(9 linear and 729 3rd-order terms) , and 9-tap linear with 3-tap 3rd-order (9 linear, 
and 27 3rd-order terms). For the linear equalizer, the 16-QAM system must be 
operated at a large backoff for low P b . However, the 3rd-order nonlinear equalizers 
can operate at 6 dB backoff and still maintain a low P b . 

The 9-tap 3rd-order equalizer has approximately a 0.5 dB better performance 
than the 9-tap linear with 3-tap 3rd-order equalizer, however the former is an 
impractical device. The significant improvement for the nonlinear equalizers is in 
link margin. 5 The E b /No for the curves in Fig. 3.10 is measured at the receiver. 


5 Link margin is defined as the energy in the received signal above a pre-defined level. 

60 





Consequently, a system with X dB backoff will have X dB less Eb/N 0 at the 
receiver than a system with 0 dB backoff. Therefore, the 9-tap linear, 3-tap 3rd- 
order equalizer at 6 dB backoff gives approximately a 3.5 dB improvement in link 
margin compared to the 9-tap linear device at 10 dB backoff. 

The nonlinear equalizer improvement in P 6 for 16-QAM versus PSK systems 
is attributed to two dominant factors: 16-QAM operates at a higher Eb/No than 
theJ-’SK systems and QAM is subject to constellation warping [10] so that the 
nonlinear effects of the HPA are more si gnifi cant. In contrast to constellation 
warping, PSK suffers only a constellation rotation which is reversed by the phase 
synchronizer. Note also that the link penalty for the PSK systems considered is 
not nearly as severe compared to the QAM system. 

3.3 FSE-Volterra Equalizer 

Fig. 3.11 illustrates the noiseless received signal spectrum and receive filter 
response for a QPSK system as in Fig. 3.1. The receive filter is matched to 
the transmitter which has a rectangular impulse response, and the remaining 
system parameters are identical to those for the systems of the previous section. 
As is evident from the figure, the receive filters of the previous section, which 
were matched to the transmitter, are severely mismatched to the channel due to 
presence of the pre-filter, TWT, and post-filter. Thus, the signal to noise ratio at 
the output of the rectangular receive filter is suboptimal. Biglieri et al. [42] have 
derived the optimal linear receive filter for nonlinear channels. Unfortunately, the 
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Figure 3.11. Received signal spectrum, rectangular filter response, and 
pseudo-matched filter response for QPSK. 

channel statistics including state transition probabilities must be known and are 
not easily obtained. 

Also shown in Fig. 3.11 is the response of a filter equivalent to the combined 
response of the pre-filter, post-filter, and transmit filter ( in this case the filters 
are Butterworth, Butterworth, and rectangular, respectively). This filter will be 
called the pseudo-matched filter. As is evident, the pseudo-matched filter matches 
the received signal spectrum much better than the rectangular receive filter. In 
Section 3.3.3, the performance of the pseudo-matched filter followed by a T-spaced 
equalizer and Volterra equalizer is compared to that of the FSE-Volterra equalizer 
for 16-PSK. 
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Alternatively, an FSE has the capability of an analog filter so that it may 
be configured as the best linear receiver [25]. Also, the FSE can compensate 
for sample timing offset. Konstantinides and Yao [24], using a state machine 
description, have developed a method for obtaining the correlation matrix R' and 
cross correlation vector p' of a nonlinear satellite communications channel. The 
optimum FSE tap weights may then be obtained from (3.4). For the case of 
unknown channel statistics an adaptive FSE can adaptively realize the optimum 
linear filter [25]. The FSE is equivalent to a matched filter followed by a T-spaced 
tapped delay fine. 

The following section discusses in greater detail the salient features of FSE 
equalizers. Next, the receive filter is replaced with an FSE so that the satellite 
receiver consists of an FSE followed by a Volterra equalizer. The operation of this 
structure is then discussed. Finally, the performance of the FSE- Volterra receiver 
is evaluated for various M-PSK scenarios. 


3.3.1 FSE Background 


The following FSE discussion follows that of [25]. First, consider the symbol- 
spaced (synchronous) equalizer with noiseless input samples x (nT + r) and taps 
Cfc, where r represents sample timing error. The output spectrum of the equalizer 
is given by 


Zr(f) = J2°k e j2nfT J2 x (/ + fj ex P 




r 


= c T (f)x T (f), 


(3.27) 
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where Xr(f) is the folded (aliased) spectrum of the input signal, and Cx(f) is 
the frequency response of the equalizer tap weights. Since the spectrum of the 
equalizer filter is periodic, Ct(/) = Cr{f + f/T), the synchronous equalizer can 
only act to modify the folded spectrum Xt{J), as opposed to directly modifying 
X(f)e~ j2 ** T . Thus, if, because of severe phase distortion and a poor choice of 
r, a null is created in the aliased part of the spectrum then the equalizer can 
only synthesize a large gain in the region. Unfortunately, this leads to a severe 
performance degradation because of noise enhancement. 

The fractionally-spaced equalizer is similar to the transversal filter; however 
the- tap spacing is given by T' < T such that the input signal x(t) into the 
FSE is sampled at greater than twice the highest frequency component. For 
practical purposes, V is chosen to be a rational fraction of T (it is typical to 
choose T' = T/2). Since data decisions are made at the symbol intervals, the 
output of the FSE is sampled at the rate 1/T. The output spectrum of the FSE 
prior to symbol-rate sampling is given by 


z* (/) = IT c * e j2wfT Y, x + exp 


j 



T 


« c r (f)Xr(f). 


(3.28) 

(3.29) 


As in the previous case, the equalizer spectrum is periodic, C^if) = C T -{f + 
l/T'). However, the sample rate l/T' is greater than twice the highest frequency 
component in X(f). Consequently, only the l = 0 term in (3.28) is nonzero in the 
interval J/| < 1/(2 T f ).’ Thus, the FSE output spectrum becomes 

M/) = <V |/| <^. (3.30) 
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Figure 3.12. FSE-Volterra receiver. 

From (3.30) it is clear that modifies X(f)e~^ 2vfr before aliasing and C T > (/) 

can compensate for sample timing offset. 

After sampling the equalizer output at the rate 1 / T the periodic spectrum is 
given by 


l 


l 


2r(/) = £C^ / + - X / + - exp 


■'V + t) t 


(3.31) 


It is evident that (3.31) is the sum of equalized components while (3.27) is the 
sum of equalized aliased components. Thus, the FSE has the capabilities of an 
analog filter and can be configured as the optimum linear receiver. 


3.3.2 FSE-Volterra Receiver 

The FSE-Volterra receiver is shown in Fig. 3.12. The input signal x(t) to the 
receiver is the output of the channel plus noise and is sampled at the rate 1 /T'. 
The receive filter (e.g., as in Fig. 3.1) is replaced with an FSE. The output of the 
FSE y(n'T') is sampled at the symbol rate and is input to the Volterra equalizer. 
The detector then outputs an estimate dn of the transmitted symbol d n . 

Adaptive setting of the FSE weights is obtained via the LMS algorithm where 
tap updates occur at the rate 1/T. Since the FSE is a linear adaptive filter, and 
the nonlinear channel output characteristics can be expressed in the form of a 
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correlation matrix R 7 and cross-correlation vector p 7 , then the previous discussions 
on the LMS convergence apply. 

The adaption strategy is as follows. First the Volterra tap weights are initial- 
ized to an impulse function (i.e., the center linear tap is set to unity and all others 
set to zero). The initial FSE tap weights are set such that the FSE is matched 
to the transmit filter. The FSE is then adapted via the LMS algorithm until the 
MSE, at its output, is no longer decreasing. The FSE adaption is then turned 
off. Finally, the Volterra equalizer is adapted until the MSE, at its output, is no 
longer decreasing. 

3.3.3 FSE- Volterra Receiver Probability of Error Performance 

Probability of bit error performance curves for the FSE- Volterra receiver of 
Fig. 3.12 is shown in Figs. 3.13-3.15. For each of the curves, the number of 
equalizer taps was chosen such that increasing the number of taps did not provide 
performance improvement. Also, the performance of the FSE- Volterra equalizer 
is compared to the synchronous equalizer and FSE. The system parameters are 
the same as for Fig. 3.4. 

The computer simulations which evaluated the following systems used discrete- 
time signals, where the sample rate was four samples per symbol. In the case of 
rectangular transmit and receive filters, this resulted in a sampling-phase error 

at the detector causing a loss of approximately 0.15 dB. That is, the optimum 

* 

sampling point at the receive filter output was in error such that the signal to 
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Figure 3.13. FSE-Volterra QPSK probability of bit error performance. 



Figure 3.14. FSE-Volterra 8-PSK probability of bit error performance. 
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Figure 3.15. 16-PSK FSE-Volterra and pseudo-matched filter probability of er- 
ror performance. 

noise ratio was decreased by 0.15 dB. For the case of the pseudo-matched filter 
the sampling error loss was approximately 0.18 dB. The curves in Figs. 3.13-3.15 
show the error rates obtained directly from computer simulation and have not 
been adjusted for sampling phase error. 

Fig. 3.13 summarizes the QPSK probability of bit error performance. The 
FSE-Volterra equalizer shows a small gain in performance relative to the FSE, 
approximately 0.2 dB at Pb = 10 -5 . Also it is evident that the FSE and FSE- 
Volterra obtain a significant improvement in performance (~1 dB at Pb = 10 -5 ) 
compared to the 5-tap linear equalizer following the receive filter. In this case, 
the FSE-Volterra performance is approximately 0.3 dB away from theory. 

For 8-PSK, Fig. 3.14 shows that the FSE-Volterra equalizer gain relative to 
the FSE is better than for the QPSK case. The FSE-Volterra Pb performance is 
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within 1 dB from theory and improves upon the FSE by 0.5 dB. The FSE gains 
1 dB relative to the synchronous equalizer. 

Fig. 3.15 illustrates that the FSE-Volterra Pj, performance for 16-PSK is im- 
proved approximately 3-dB relative to the FSE equalizer and is 2-dB away from 
theory. The FSE gains over 1 dB relative to the synchronous equalizer. For this 
case, it was found that the communications link was incapable of providing error- 
less communication , even without noise. Thus, linear equalization provided a sig- 
nificant performance improvement. In addition, the noise level was low enough 
such that Volterra equalizer could effectively remove some of the nonlinear distor- 
tion without severely enhancing the noise. 

Also, the pseudo-matched filter followed by a 9-tap T-spaced equalizer and 
3-tap 3rd-order Volterra equalizer gives performance within 0.4 dB of the FSE- 
Volterra equalizer. When the pseudo-matched filter curve is adjusted in order 
to compensate for sampling-phase error, the difference in performance is approx- 
imately 0.2dB. This illustrates that when the response of the channel filters is 
known in advance, the FSE equalizer is not necessary (i.e., a pseudo-matched fil- 
ter with a T-spaced equalizer gives similar performance). 

3.4 Chapter Summary 

This chapter has reviewed adaptive linear filter theory relevant to the un- 
derstanding of the adaptive Volterra equalizer. The convergence of the adaptive 
(LMS algorithm) Volterra equalizer was analyzed. It was found that the conver- 
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gence rate is slower for the Volterra equalizer than for a linear device with an 
equivalent length tapped delay line. Consequently, a multiple-step size adaption 
method was employed which improved the convergence rate while maintaining 
good MSE performance. 

The MSE and probability of bit error performance was characterized for the 
adaptive Volterra equalizer. It was found that a significant improvement in MSE 
did not necessarily translate to a commensurate improvement in probability of 
error. Also, an FSE- Volterra equalizer gives improved performance relative to 
the Volterra equalizer when the receive filter is not matched to the channel. The 
Volterra and FSE- Volterra equalizers give significant performance improvement 
for multi-level (i.e., QAM) and modulation schemes which require high signal to 
noise ratios. Also, the pseudo-matched filter followed by a T-spaced equalizer and 
Volterra equalizer gives similar performance to the FSE- Volterra equalizer. 
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Chapter 4 

MLSD RECEIVERS FOR NONLINEAR SATELLITE CHANNELS 


Forney [23] has shown that the optimum receiver for a Unear channel with 
ISI is a whitened matched filter followed by a nonlinear processor known as the 
Viterbi algorithm [26]. A maximum-Ukelihood receiver for binary sequences over 
bancUimited nonhnear channels was derived by Campbell [21]. Benedetto et al. 
[22] derived the optimal receiver for a nonlinear bandUmited satelhte channel and 
arbitrary modulation formats. Although the approach for deriving the receivers 
in [22] and [21] differed, both have a similar structure; a bank of matched filters 
followed by a Viterbi detector. In [21] a performance analysis was performed 
for a typical satelhte system. It was found, for binary sequences, the MLSD 
receiver required an excess of approximately 1 dB of downUnk SNR to equal the 
performance of binary transmission over an AWGN channel. The performance 
of the receiver derived in [22], however, was not evaluated for a specific satellite 
channel. In this chapter, the performance of a matched filter bank Viterbi detector 
(MFB-MLSD) receiver, as in [22], is evaluated for QPSK and 8-PSK satellite 
channels. 

Because of the matched filter bank, the MFB-MLSD receiver may be high 

in complexity. Consequently, the performance of a suboptimal receiver, a single 

receive filter followed by Viterbi detector (SF-MLSD), is also studied. The SF- 

MLSD receiver represents the typical receiver where there is a single receive filter. 

Since the channels of the previous chapter employed a single receive filter, the 
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SF-MLSD probability of error performance serves as a lower bound to that of the 
Volterra and linear equalizers, where the linear equalizers follow a fixed receive 
filter. Furthermore, the MFB-MLSD lower bounds the probability of error perfor- 
mance of the SF-MLSD receiver and the FSE- Volterra equalizers of the previous 
chapter. 

In order to minimize the effects of nonlinear distortion, it is typical to trans- 
mit rectangular pulse shapes through satellite channels. However, because of the 
satellite filters, this may result in a net signal loss. It may be advantageous to 
transmit bandlimited signals, such as signals with a square root raised-cosine pulse 
shape. It is found that a nonlinear bandlimited satellite channel which employs 
square root raised-cosine pulse shapes with a SF-MLSD receiver gives little per- 
formance loss relative to the same system with an MFB-MLSD receiver. Also, the 
performance of the MFB-MLSD receiver for the case of square root raised-cosine 
signals indicates that non-constant envelope signals through nonlinear satellite 
channels are capable of giving performance close to that of the AWGN channel. 

The complexity of the MFB-MLSD receiver, specifically the filter bank, grows 
exponentially with the memory of the communications channel. Consequently, 
both rectangular and square root raised-cosine pulse shapes are considered, with 
the goal of minimiz ing the channel memory. It is found that for high excess 
bandwidth (i.e., 100%) the square-root raised cosine pulse shape results in minimal 
channel memory. Lower amounts of excess bandwidth result in larger memory. 
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The organization of this chapter is as follows. First, two representative nonlin- 
ear satellite channels and associated parameters are described. These systems will 
be referenced throughout this chapter. Next, an SF-MLSD receiver is described 
followed by a brief description of the Viterbi algorithm. The Viterbi algorithm 
discussion will be useful when describing the relationship between the Viterbi al- 
gorithm path metric calculations and the matched filter bank of the MFB-MLSD 
receiver. Pulse shaping and its effect on channel memory is then discussed. Next, 
a log-likelihood development of the MFB-MLSD receiver, as in [22], is reviewed. 
Next, the probability of error performance of these MLSD receivers is evaluated 
via computer simulation and calculation of d min . The method of calculating d min , 
as in [22], is also described. The calculation of dmin and the computer simula- 
tion employ state machine models of the communications channels as described 
in Chapter 2. 

4.1 Satellite Channel Configurations 

The discussion and analysis in this chapter will reference two representative 
nonlinear satellite communication systems. The first system is similar to those of 
the previous chapter which employed rectangular transmit and receive filters. The 
second system employs square root raised-cosine filters. The first system has the 
disadvantage of signal loss due to the satellite filters. For the second system, it is 
assumed that the transmitted signal spectrum fits sufficiently inside the satellite’s 
filters. Thus, for this case the signal is unaffected by the pre- and post-filters and 
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there is no signal loss. For both systems, when an SF-MLSD receiver is employed, 
the receive filter is matched to the transmitter. The systems are described in more 
detail in the following paragraphs. 

System I is identical to the channel studied in the previous chapter. That 
is, the satellite pre- and post-filters are 6th order butterworth with a 3 dB cutoff 
frequency of 0.75 R s , where the symbol r R s = 1/T , and T is the symbol interval. 
The TWT is driven at 0 dB backoff. The transmit filter has a rectangular pulse 
shape (i.e., hr(t) = 1, 0 < t < T). The study of this system is interesting since the 
performance results of the SF-MLSD and MFB-MLSD receivers may be compared 
to the performance results of the previous chapter. 

System II employs a square root raised-cosine pulse shape at the transmitter 
with 100% excess bandwidth. For this channel, the satellite pre- and post-filters 
are replaced with wires. The TWT is driven at 0 dB backoff. System II is 
unrelated to System I. For example, it is not assumed that a signal of 100% excess 
bandwidth is unaffected by filters with a 3 dB cutoff frequency of 0.75 R s . It is 
interesting to study this system since the performance of the MFB-MLSD receiver 
will indicate the performance of an optimum receiver for non-constant modulus 
signals through a nonlinear satellite channel. The performance of the SF-MLSD 
receiver for this channel will indicate the performance of a practical receiver for 
this system configuration. 
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Figure 4.1. Satellite communication system with SF-MLSD receiver. 

4.2 Single Filter MLSD Receiver 

The SF-MLSD receiver is shown in Fig. 4.1. The input to the transmit- 
ter is the information sequence {a n }, where a n . € {0, 1, . . . , M - 1} , and M is 
the symbol set size. The waveforms h(t,a n ,a , n ) are defined as in Section 2.3. 
The inputs to the detector { 7 „} are the sampled outputs of the receive filter 
hji(t). The noise w(t) is assumed to be an AWGN process. Also, the receive 
filter h,R(t) is matched to the transmitter, as is typical in satellite communica- 
tion systems. If the information sequence cxn = • • • i&n-N+i} is de- 

termined to be the maximum-likelihood transmitted sequence given that the se- 
quence Jn = (7n,7n-i) • • • ,7n-Ar+i} was received, then the output of the Viterbi 
detector is a n _ d = a n _<f, where d is the decoding delay. 

The following subsections discuss two important elements associated with 
the SF-MLSD receiver of Fig. 4.1. First, the Viterbi algorithm is discussed with 
respect to the nonlinear satellite channel. The complexity of the Viterbi algorithm 
is determined by the number of states assumed by the communications channel 
and the path memory. However, the path memory is directly proportional to the 
memory of the communications channel. Consequently, the effect of the pulse 


75 





shape on the communications system memory is then discussed. This includes 
evaluating the communication system memory for rectangular and raised-cosine 
pulse shapes. 

4.2.1 The Viterbi Algorithm 

The Viterbi algorithm [26] was developed for decoding of convolutional codes 
and later employed for detecting of ISI sequences [23, 43, 44]. Here, the Viterbi 
algorithm is discussed in the context of the satellite communications model of Fig. 
4.1. 

Suppose that the sequence of N (N large) detector inputs is received and 
the communication system has a memory of L symbols. The receive filter output 
sample is given by 

In = §n ■+■ Vn (4.1) 

where s n is the output of the satellite ch ann el filtered by the receive filter and 
sampled at time nT. The noise sample r) n is obtained by filtering the noise process 
w(t) and sampling. Since in this work hx(t) is either a rectangular filter or a 
square root raised-cosine filter, the noise samples 7j n at the output of the receive 
filter are independent. 

The objective of the MLSD receiver is to choose the sequence 

a = {On, On-l) ttn-JV+l} (4-2) 

which maximizes the conditional probability density (or likelihood) function 

p[ tM = p[ 7n, 7n-l, • • • , 7n-JV+l|On, On-1, • • • , On-TV+l] (4.3) 
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Figure 4.2. Trellis diagram for communications system with M = 2 and L = 2. 
which is the probability of the receive filter output sequence conditioned on the 
input sequence. This is known as the maximum-likelihood criterion. Because the 
noise samples at the output of the receive filter are independent, the likelihood 
function (4.3) can be expressed as a product of marginal densities 

N - 1 

P[7l a ] — II P(7n-fc|On-fc, , On-fc-Z,) (4-4) 

fc =0 

Since the noise process w(t) is assumed to be AWGN, the objective of the de- 
tector reduces to choosing the sequence a which minimizes the Euclidean distance 
to the received sequence 7. That is, choose a such that the squared Euclidean 
distance 

d 2 (7, s) = 53 (7 n-k ~ s (a n _ fc , (p n -k)) 2 (4.5) 

fc =0 

is minimized, where s(on, ip n ) is the noiseless input to the detector given that the 
data sequence a was transmitted and the state of the channel is a n with input a n , 
and (T n — An— 2) • • • j ^n— l} • 

As discussed in Section 2.3 (FSM model), a communications channel with a 
memory of L symbols has M L states. Such a system may be described as a trellis 
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with M paths entering and leaving each state. For example, Fig. 4.2 illustrates 
a 4-state trellis with M = 2 and L = 2. The labels on each branch of the trellis 
indicate the output of the satellite channel followed by the input. For example, 
the label on the branch from state 00 to state 10 indicates that when the system 
is in state 00 and an input symbol 1 is transmitted, the system goes to state 10 
with an output -0.9 volts. 

-The Vitexbi decoder utilizes the trellis structure to efficiently provide a max- 
imum-likelihood estimate a of the transmitted sequence from which an estimate 
<in-d of the transmitted symbol is derived. At each stage k of the trellis, a branch 
metric 

b{p-n—ki (Prt—k) ~ (pin—k s(&n — fci *Pn—k ) ) (4.6) 

is calculated for each branch exiting a particular state at stage k of the trellis. 
The branch metric is then added to the path metric p m ^ corresponding to state 
m and stage k. Next, the path entering each state with the smallest path metric 
is retained and the remaining paths are discarded. Since for two or more paths 
entering a particular state the paths will be identical from that point forward, 
a maximum-likelihood decoder may discard all paths entering a state, except 
the path with the smallest metric, without loss in performance. The number of 
decoding stages, d, required for near optimum performance is approximately 5 L 
[17]. After d decoding stages the path with the minimum path metric determines 
the maximum likelihood estimate a n _ d . 
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Table 4.4. State table for System I with rectangular receive filter. 



0-n 

output real 

output imag. 

0 

B 

El 

a 

B 

B 

B 

-0.735 

-0.756 

0 

0 

El 

D 

0 

0 

B 

0.487 

-0.809 

0 

B 

El 

B 

0 

0 

B 

-0.804 

0.534 

0 

0 

D 

B 

0 

B 

B 

0.517 

0.515 

0 

0 

D 

B 

0 

0 

B 

-0.849 

-0.743 

0 

0 

B 

0 

0 

0 

B 



0 

0 

H 

B 

0 

B 

B 

-0.787 

-0.810 

0 

0 

B 

0 

1 

0 

B 

-0.434 

-0.841 

0 

0 

a 

B 

B 

0 

B 


-0.371 

0 

Q 

B 

B 

B 

B 

B 

-0.461 

-0.421 

0 

i 

a 

B 

B 

B 

B 

-0.701 

-0.775 

1 

0 

a 

0 

B 

B 

B 

-0.737 

-0.748 

0 

Q 

El 

0 

B 

B 

B 

-0.730 

-0.761 

0 

El 

o 

B 

B 

B 

B 

-0.744 

-0.747 


4.2.2 Pulse Shaping and Channel Memory 

Because the satellite channel is bandlimited, this leads to a communication 
system with memory. The channel memory may be assessed by looking at the 
state table description of the communications channel. Table 4.4 contains a par- 
tial listing of the state table for a satellite channel with rectangular receive and 
transmit filters as in System I. Since this table is for QPSK modulation, the sym- 
bols, On, and state, <T n ={a n _ 1 , a n _ 2 , . . . , a„_6} , entries are elements from the set 
{0, 1,2,3} . The last two columns indicate the real and imaginary discrete- time 
outputs for the given state and input. 

T his table illustrates the effects of the three past and three future symbols on 
the center symbol, a n -3- The symbol a „_3 determines the quadrant for the output 
of the particular input-state combination, and as in section 2.3.2 it will be called 
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the “punctual symbol.” It is evident from the table that the input-state entries 
in the table with a „_3 = 0, 1, 2, 3, map to quadrants III, IV, II, I, respectively. 
The first row indicates that the output for an information symbol 0 preceded by 
three information symbols of 0 and followed by three information symbols of 0 is 
—0.735 — j'0.756. Row number five illustrates that the output for an information 
symbol 0 immediately preceded by an information symbol 1 is —0.849 — j0.743. 
The distance between these points is 0.115. Using the first row as a reference, 
then the preceding information symbol of 1 affected the 0 .input-state (row no. 1) 
by 101og 10 (0.115/V®:) 2 = —18.8 dB, where the symbol energy E s = 1 . 

Studying the complete state table, the worst case effect of past and present 
symbols may be determined. Table 4.5 lists the worst case effect for past and 
future symbols, where L p indicates a symbol L p symbols in the past and Lf is 
defined similarly for future symbols. To measure the effect of a given symbol a, on 
the punctual symbol a P , we fix the values of the symbols surrounding a, (including 
Op), then vary a* over all possible symbols, {0, 1, . . . , M — 1}. As we vary a,, we 
measure the maximum difference, m^, between all possible pairs of the M output 
levels corresponding to the Cj’s. Then, the effect Bjj on the punctual symbol is 
defined as 

£ // = 101og 10 (^|=) (4.7) 

The largest effect (i.e., worst case effect) is then obtained by considering all pos- 
sible combinations of surrounding symbols, and choosing the maximum to <*. This 
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Table 4.5. Channel memory for System I. 


L P 


1 

< -10.6 dB 

2 

< -16.2 dB 

3 

< -26.6 dB 

V 


1 

< -3.72 dB 

2 

< -29.1 dB 


Table 4.6. State table for raised-cosine filtering 


in 

state 

output real 

output imag. 

0 

0 

0 

0 

0 

-0.735 

-0.749 

0 

0 

1 

0 

0 

0.621 

-0.710 

0 

0 

2 

0 

0 

-0.774 

0.640 

0 

0 

3 

0 

0 

0.538 

0.624 

0 

1 

0 

0 

0 


-0.768 

0 

2 

0 

0 

0 

-0.745 

-0.709 

0 

3 

0 

0 

0 

-0.686 

-0.697 

0 

0 

0 

1 

0 


-0.769 

0 

0 

0 

2 

0 

-0.747 

-0.709 

0 

0 

0 

3 

0 

1 

-0.681 

-0.696 

1 

0 

0 

0 

0 

-0.732 

-0.759 

0 

0 

0 

0 

1 

-0.738 

-0.754 
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table illustrates that the significant channel memory is approximately two past 
symbols and one future symbol. 

Table 4.6 contains a partial state table listing for a satellite communication 
system where the receive and transmit filters are square root raised-cosine as in 
System II. In this case, the punctual symbol is <!„_!. A lesser amount of excess 
bandwidth results in increasing the channel memory. For example, the square 
root raised cosine pulse shape is illustrated in Fig. 4.3. Because of the higher side 
lobes, a pulse with a smaller amount of excess bandwidth is affected more strongly 
by the nonlinearity and consequently influences neighboring symbols by a greater 
amount. Table 4.7 lists the worst case effect for past and future symbols for this 
channel. The significant channel memory is one past and one future symbol. In a 
similar fashion, it has been determined that the memory for a system using pulses 
with excess bandwidth of 50% is two past and two future symbols. 

State tables similar to Tables 4.4 and 4.6 with Lj = 1, L p = 2, and Lj = 1, 
L p = 1, respectively, are used to simulate the performance of System I and System 
II in Section 4.4. This will be discussed further in Section 4.4. 

4.3 Matched Filter Bank MLSD Receiver 

The MFB-MLSD receiver is shown in Fig. 4.4. As in the SF-MLSD case, the 
input to the transmitter is the information sequence {a*}, where a„ € {0, 1, ... , 
M — 1} and M is the symbol set size. The receiver consists of a filter bank with 
a filter matched to each chip h(t, a„, a n ) out of the satellite channel. 


82 



2 


3 


Symbols 


Figure 4.3. Square root raised-cosine pulse shapes. 


Table 4.7. Channel memory for raised cosine filtering 












Figure 4.4. Satellite communication system with MFB-MLSD receiver. 
Below, the MFB-MLSD receiver is developed using log-likelihood functions. 
Next, the relationship between the Viter bi algorithm path metrics and matched 
filter bank is described. 

4.3.1 The Matched Filter Bank MLSD Receiver Derivation 

The log-likelihood development of the MFB-MLSD receiver follows that of 
[22, ch. 10]. The objective is the maximum-likelihood detection of the finite 
sequence of symbols 

a = {aw_ i, att- 2 , • • • > oo} , (4.8) 

where in order to simplify the notation in the following development, n as in (4.2) 
corresponds to N — 1. The detection process is based on the observation of the 
waveform 

N - 1 

r(t) = ^h(t- nT, a„, a n ) + w(t) (4.9) 

n=0 

where N » L, L is the symbol memory, the state a n = {on_i, . . . , <!„_£,}, and 

w(t) is an AWGN process. The chips h(t, On, a n ) are defined as in Section 2.3. 
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The log-likelihood function, may be derived with methods similar to those in [45, 


ch. 4] for the “M-hypothesis” case and is given by [22] 6 

2 f r NT 1 1 r NT 

li= N^ R '\l S *® r ® dt \ ~ Hq fo dt ' ^‘ 10 ^ 

where Si(t) is the noiseless transmitted waveform given that the sequence a* = 
|a^_ a a i N _ 2 , . . . , Aq} was sent, and is given by 

Si (t) =f^h(t- nT , <4 <). (4.11) 

71=0 

The decision rule involves setting a = a* whenever the log-likelihood function U 

is maximum for i = k. Recalling that h(t, a n , a n ) is nonzero only over the interval 

[0 ,T), substituting (4.11) in (4.10), and after straightforward manipulations, the 

likelihood function becomes 

2 f N_1 r(n+i)T / . 1 

>< = t 0 k {ZL 


1 ^ ^ /*(n-f-l)X . > \ i2 

-j^E o /„ T dt (4.12) 

Defining the functions 

r(n+l)T . ... 

Z(o^, O = J h* (t - nT, a\, r(t)dt, and (4. 13) 

= 2 dt, (4.14) 

and substituting into (4.12) yields 

(415) 


5(a^,<r4 is the energy in the chips h(t,a l n ,a *)• Z(al i ,a i n ) can be obtained as 
the response, sampled at time (n + 1)T, of a filter matched to h (t, a l n , a l n ). This 


®The M in “M-hypotheses” is not related to the signal set size. 
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implies that the number of matched filters required is M L+1 , which is the number 
chips h ( t , a n , cr n ). 

Multiplying l, by —No, we obtain a new likelihood function 

k = -2R { £ z( 4, <)} + X £(<, <)■ (4.16) 

l n= 0 J n=0 

The decision rule involves setting a = a* whenever the log-likelihood function 
/», given by equation (4.16), is minimum for i — k. A brute force method for 
mifiimization of (4.16) would require evaluating all M N choices for the sequence 
a*. Alternatively, since the relationship between the elements of the sequence a* 
may be described by a finite-state machine, the Viterbi algorithm can efficiently 
provide the maximum-likelihood estimate a. A program which calculates the 
path metrics according to equation (4.16) based on a state table description of 
the channel is listed in Appendix A. 

4.3.2 The Matched Filter Bank and Viterbi Algorithm 

As indicated above, the Viterbi algorithm can efficiently provide a maximum- 
likelihood estimate of the transmitted sequence a. Also, the Viterbi algorithm was 
previously described in the context of the SF-MLSD receiver. In the case of the 
MFB-MLSD receiver, in order for the Viterbi algorithm to properly detect the 
sequence a, it is necessary for the appropriate matched filter output sample to be 
used in the calculation of the path metrics. From the previous discussion of the 
Viterbi algorithm the branch metrics were given by equation (4.6). In this case, 

the branch metric for the branch leaving state cr x n _ k with input a 1 n _ k is given by a 
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Figure 4.5. Communication system simulation model, 
similar expression 

= (4.17) 

where Z(a 1 n _ k) cr* n _ k ) is the output of the filter, sampled at time (n + 1)T, matched 
to h(t, c l n _ k ). That is, the metric for the branch leaving state a l n _ k with input 
a\_ k uses the output of the filter matched to the corresponding chip. 

4.4 Probability of Error Performance 

The probability of error ( P e ) performance for the SF-MLSD and MFB-MLSD 
receivers is evaluated by calculating the minimum distance between channel se- 
quences (dm j n ) and by computer simulation. The minimum distance between chan- 
nel sequences is derived from the state table description of the channel. For the 
MFB-MLSD receiver a program which reads the state table and calculates d mm is 
listed in Appendix A. 

Fig. 4.5 illustrates the simulation model. For the systems with an SF-MLSD 
receiver, the FSM model, as described in section 2.3.2, is used to generate the 
channel outputs. Discrete noise samples are then added to the channel outputs. 
Since in the SF-MLSD case the output of the channel consists of one sample per 
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symbol, n = n in the figure. The MLSD receiver uses the same state table, as 
in the transmitter, for calculating the path metrics. The receiver then provides 
the maximum-likelihood estimates a n -d- For the systems with an MFB-MLSD 
receiver, the FSM model, as described in section 2.3.3, generates the oversampled 
channel outputs. Discrete time noise samples r) n > are then added to the channel 
outputs. The MFB-MLSD receiver then updates the matched filter bank output 
ah the end of each symbol period, calculates the path metrics, and provides the 
estimate a n -d ■ As in the previous case, the MFB-MLSD receiver uses the same 
state description of the channel as the transmitter. A program for simulating 
the nonlinear bandlimited satellite channel, as in Fig. 4.5, with an MFB-MLSD 
receiver is listed in Appendix A. 

4.4.1 Calculation of d m j n 

An upper bound for the probability of symbol of error may be obtained using 
d m in, defined as the minimum Euclidean distance between channel sequences. The 
nonlinear satellite channel is equivalent to a nonlinear transmitter with memory. 
This nonlinear transmitter nonlinearily encodes the information sequence, and 
the distance between the resulting “codewords” determines the probability of er- 
roneously detecting a received sequence. As in the case of error control codes, the 
probability of error for the nonlinear ISI channel is bounded above by the proba- 
bility of choosing an incorrect sequence that lies at the minimum distance away 
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from the correct sequence: 7 

(4.19) 

where a 2 is the variance of the noise at the input to the detector. Also, (4.19) 
assumes independent Gaussian noise samples at the input to the detector. 

For the nonlinear satellite channel with memory, finding d min requires exhaus- 
tive search between all pairs of received symbol sequences. The distance structure 
from an all zeros path is not sufficient because, unlike the linear situation, the dis- 
tance spectrum is not identical for every sequence. Since for the typical satellite 
channel the memory is short (e.g., Tables 4.6 and 4.7), a brute force approach to 
calculating d m i n may be possible. With higher order modulations (i.e., large M), 
however, the brute force method may still be prohibitive. 

An efficient algorithm for calculating d m j n is given by Saxena [46] and Mulligan 
and Wilson [47], and a step by step description is given in [22, ch. 10]. A 
brief description of the algorithm is given here. For a communication system 
with modulation order M and memory L, the algorithm requires updating of 
an M L by M L matrix D^ n \ The elements 8j^ of the matrix represent the 
squared minimum distance between all pairs of paths diverging from some initial 

7 The bound is precisely given by 


Pe<2>«< 3(£), (4-18) 

d 

where Nd is the average number of sequences at a distance d. For large values of signal to noise ratio, 
the dominant error term is the one containing dmin . We conjecture that Nd min is on the order of 1 and 
possibly 2 as for M-PSK modulation, where Nd min is the average number of sequences at a distance 
dmin ■ 
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Table 4.8. Dmin values. 



^min ( r Ct) 

G^min (sqrc) 

SF-MLSD QPSK 

1.125 

1.327 

MFB-MLSD QPSK 

1.222 

1.39 

SF-MLSD 8-PSK 

0.566 

0.684 

MFB-MLSD 8-PSK 

0.639 

0.747 


state and passing at the nth time instant through states Si and Sj. The main 
diagonal represents paths which have merged, and the off diagonal terms represent 
minimum distances between diverged paths which have not yet merged. 

The elements 6-V of are set to the mini mum distance between all pairs 
of paths (Si, Sj) which have diverged from the same initial state and reaching the 
states Si and Sj in one step. The algorithm then proceeds iteratively where at 
each step (i.e., n = 2, 3, . . .) the elements of are updated according to the 
minimum distances of paths which have diverged and have reached states S, and 
Sj at time n. The entry 6-^ is obtained by finding the predecessor path from the 
M 2 paths at the (n — l)th time instant which pass through the states Si and 
Sj at the nth time instant. 6^ is then the minimum squared distance obtained 
by accumulating the incremental distance from the predecessor states to states 
Si and Sj. When the off diagonal terms are all greater than the main diagonal 
terms then there are no paths which can merge which have a smaller minimum 
distance than the already merged paths. This is where the algorithm stops and 
the minimum distance is set to the smallest diagonal entry dmin = min (4r>). 

The minimum distances were calculated for systems with SF-MLSD and 
MFB-MLSD receivers and are listed in Table 4.8 with rectangular (ret) and square 
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root raised-cosine (sqrc) filters. As expected the MFB-MLSD receivers have d mm 
greater than the corresponding SF-MLSD receivers. Also, the systems with square 
root raised-cosine filters have larger d m i n than the corresponding systems with 
rectangular filters. 

4.4.2 Computer Simulation Results 

Figs. 4.6 and 4.7 illustrate the performance of SF-MLSD and MFB-MLSD 
receivers for System I, respectively. The symbol error performance for Q-PSK and 
8-PSK are plotted and compared to the upper bound obtained by evaluating (4.19) 
using the appropriate entry from Table 4.8. For System I with QPSK and 8-PSK 
modulation, the SF-MLSD receiver is approximately 0.75 dB and 1 dB below the 
upper bound and 1 dB and 1.5 dB above theory at P e = 10~ 5 , respectively. For 
System I with the MFB-MLSD receiver, the performance is approximately 0.5 dB 
above theory for both QPSK and 8-PSK modulation. 

Fig. 4.8 compares the performance of the SF-MLSD receiver to the perfor- 
mance obtained from an adaptive equalizer followed by a symbol by symbol de- 
tector for System I. Other than the receiver (the receiver includes the receive filter 
and detector), the systems are identical, however they both have rectangular re- 
ceive filters. Recall that the 5-tap Unear equafizer was the best performing equal- 
izer (i.e., in the probability of error sense) for the systems of Fig. 4.8 and, of the 
8-PSK equalizers, the 7-tap Unear with 3-tap 3rd-order equahzer gave the best per- 
formance. In this case, the SF-MLSD receiver provides the maximum-likelihood 
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Figure 4.6. Performance of SF-MLSD receiver for System I. 
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Figure 4.7. Performance of MFB-MLSD receivers for System I. 
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Figure 4.8. Performance of SF-MLSD receiver compared to adaptive equalizers 
for System I. 

estimate and therefore a lower bound. For QPSK and 8-PSK, P e = 10 -5 , the 
performance of a T-spaced adaptive equalizer is above the lower bound by ap- 
proximately 0.75 dB. 

Fig. 4.9 compares the performance of the MFB-MLSD receiver to the FSE- 
Volterra equalizers for System I. For QPSK modulation the FSE Volterra equal- 
izer, 10-tap T/2 FSE followed by 3-tap 3rd-order Volterra, gives performance ap- 
proximately 0.1 dB above the lower bound provided by the MFB-MLSD receiver. 
The lower bound is approximately 0.5 dB above theory. For 8-PSK, the FSE- 
Volterra equalizer, 14-tap T/2 FSE followed by 3-tap 3rd-order Volterra, gives 
performance approximately 0.75 dB above the lower bound which is 0.5 dB above 
theory. Although the MFB-MLSD receiver may be impractical (i.e., the 8-PSK 
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Figure 4.9. Performance of MFB-MLSD receiver compared to FSE-Volterra per- 
formance for System I. 



Figure 4.10. Performance of SF-MLSD and MFB-MLSD receivers for System II. 
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Figure 4.11. The affect of Viterbi detector path memory on probability of sym- 
bol error performance. 

MFB-MLSD receiver has 4,096 matched filters), these results illustrate the useful- 
ness of the MFB-MLSD receiver for bounding the performance of other receivers. 

Fig. 4.10 compares the performance of SF-MLSD and MFB-MLSD receivers 
for System II. In the case of QPSK, the MFB-MLSD receiver gives performance 
approximately 0.25 dB away from theory. For this case, the SF-MLSD receiver has 
a performance loss of approximately 0.3 dB relative to the MFB-MLSD receiver. 
In the case of 8-PSK, the MFB-MLSD receiver gives performance approximately 
0.3 dB above theory, and the SF-MLSD receiver has a performance loss approx- 
imately 0.4 dB relative to the MFB-MLSD receiver. The MFB-MLSD receiver 

results for System II illustrate that it is possible to obtain probability of symbol 
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error performance very close to theory for non-constant modulus signals through 
a nonlinear satellite channel. 

Fig. 4.11 compares the performance of Viterbi detector path memory length 
for SF-MLSD and MFB-MLSD receivers. The curves in the figure axe for System 
II. For QPSK modulation and the MFB-MLSD receiver there is a small perfor- 
mance loss for a path memory of 2 compared to a path memory of 10. The results 
are the same for 8-PSK and the SF-MLSD receiver. Similar results have been 
noticed for the other system-receiver combinations. This may be useful for sim- 
plifying the receiver implementation. 

4.5 Chapter Summary 

This chapter has described the SF-MLSD and MFB-MLSD receivers and 
their performance. The Viterbi algorithm, which efficiently provides a MLSD 
estimate, was discussed in the context of the nonlinear satellite channel with 
memory. Next, the symbol memory was characterized for a bandlimited satellite 
channel using rectangular and raised-cosine filtering. It was found that the symbol 
memory increased for systems using raised-cosine filtering with small amounts of 
excess bandwidth. Next, a log-likelihood development of the MFB-MLSD receiver, 
as in [22], was presented. Also, for the MFB-MLSD receiver, the relationship 
between the matched-filter bank and calculation of the path metrics for the Viterbi 
algorithm was described. 
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The probability of error performance of the MFB-MLSD and SF-MLSD re- 
ceivers was then analyzed. First, a method of deriving the minimum distance 
between ISI code sequences was described. Next, the minimum distance calcula- 
tions for QPSK and 8-PSK systems which employed SF-MLSD and MFB-MLSD 
receivers were reported. As expected, the systems with MFB-MLSD receivers had 
a smaller minimum distance. Also, the systems with square root raised-cosine fil- 
ters had larger d min than the corresponding systems with rectangular filters. 

The probability of error for the receivers was then obtained via Monte-Carlo 
computer simulation. First, it was verified that the performance obtained from 
computer simulations was below the upper bound calculated from d min . Next, 
the performance of the MLSD receivers was compared to systems with adaptive 
equalizers. The MLSD receivers provided a lower bound for the probability of 
error performance of the adaptive equalizers, and helped gauge the performance 
of the adaptive equalizers. Finally, the performance of the MFB-MLSD receiver 
for System II indicate that it is possible to obtain performance near theory for 
non-constant modulus signals through nonlinear satellite channels. 
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Chapter 5 

CONCLUSIONS AND FUTURE WORK 

This dissertation has evaluated the performance of several receiver-based 
methods for mitigating the effects due to nonlinear bandlimited signal distortion 
present in high data rate satellite channels. The results of this dissertation may 
serve as a baseline for the evaluation of more complex structures based on the 
receiver structures evaluated in the dissertation. In addition, the performance re- 
sults can help gauge the performance of hardware implementations of the receivers 
evaluated in the dissertation . Below, the results of each chapter are summarized 
and suggestions for future work are given. 

5.1 Summary and Conclusions 

Chapter 1 gave an overview of the dissertation and the necessary background 
regarding communications by satellite. Included was a discussion of a satellite’s 
communication subsystem for a typical communications satellite. The communi- 
cation subsystem was then simplified and a baseband equivalent model for evalua- 
tion of a satellite communications channel was presented. Next, the effects of the 
nonlinearity and bandlimiting on the digitally modulated data was demonstrated. 
Finally, a literature review of compensation methods for nonlinear distortion in 
various types of communication systems was presented. 

In Chapter 2, various discrete- time equivalent models for efficiently evaluat- 
ing the performance of the nonlinear bandlimited satellite channel were discussed. 
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First, the low-pass discrete-time equivalent model for a passband communication 
system was reviewed. Next, beginning with the low-pass equivalent Volterra series, 
a lucid development of the low-pass Volterra discrete-time model for a nonlinear 
satellite communications channel was presented. The resulting model is a polyno- 
mial expression relating an output symbol at discrete time n to the input symbols. 
Deriving this model for a particular channel is computationally intensive. 

- A FSM has previously been suggested for modeling a nonlinear satellite chan- 
nel. The last Section of Chapter 2 specifically developed a FSM model for the 
nonlinear bandlimited satellite channel. Two special cases were considered: SF- 
MLSD and MFB-MLSD receivers. In contrast to the discrete-time Volterra model, 
the FSM model is easily derived. However, it may require a large amount of mem- 
ory for storing the state machine. Despite the possibly large memory requirement, 
the FSM models are more efficient (i.e., computationally) than the Volterra model. 

Nonlinear Volterra equalizers have previously been studied for equalization 
of no nlin ear bandlimited signal distortion due to a noiseless satellite channel. 
However, the performance of adaptive Volterra equalizers has not been evaluated 
for a specific satellite channel. In Chapter 3, performance of nonlinear adaptive 
Volterra equalizers for mitigating the effects of nonlinear bandlimited signal dis- 
tortion were studied for a nonlinear bandlimited satellite communications channel. 
First, the chapter reviewed adaptive linear filter theory which is important to the 
underst anding of the adaptive Volterra equalizer. The convergence of the adaptive 
Volterra equalizer, which used the LMS algorithm for weight adaption, was then 
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analyzed. It was found that the convergence rate is slower for the Volterra equal- 
izer than for a linear device with an equivalent length tapped delay line. Con- 
sequently, a multiple-step size adaption method was employed which improved 
the convergence rate while maintaining good MSE performance. Also, the MSE 
and probability of error performance was characterized for the adaptive Volterra 
equalizer. It was found that a significant improvement in MSE did not necessarily 
translate to a commensurate improvement in probability of error. 

Also in Chapter 3, the salient characteristics of an FSE were reviewed and 
contrasted to those of the synchronous equalizer. Because of aliasing in the fre- 
quency spectrum, a synchronous equalizer is limi ted in its ability to mitigate the 
effects of nonlinear ISI. However, the FSE acts as an analog filter and can perform 
the combined functions of the receive filter and equalizer. It was found that an 
FSE- Volterra equalizer gave improved performance relative to the Volterra equal- 
izer when the receive filter is not matched to the channel. The Volterra and FSE- 
Volterra equalizers give significant performance improvement for multi-level (i.e., 
QAM) and modulation schemes which require high signal to noise ratios. Also, it 
was found that the pseudo-matched filter followed by a T-spaced linear equalizer 
and Volterra equalizer gave similar probability of error performance as compared 
to the FSE- Volterra equalizer. This latter result indicates that when the channel 
filters are known a priori the FSE equalizer is not necessary. 

In Chapter 4, the SF-MLSD and MFB-MLSD receivers were described and 
their probability of error performance was characterized for two representative 
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satellite channels. First, the Viterbi algorithm, which efficiently provides a MLSD 
estimate, was discussed in the context of the nonlinear satellite channel with 
memory. Next, the symbol memory was characterized for a bandlimited satellite 
channel using rectangular and raised-cosine filtering. It was found that the symbol 
memory increased for systems using raised-cosine filtering with small amounts of 
excess bandwidth. A log-likelihood development of the MFB-MLSD receiver, as 
im[22, ch. 10], was presented. Also, for the MFB-MLSD receiver, the relationship 
between the filter bank and the Viterbi algorithm path metric calculations was 
described. 

Next, in Chapter 4, a method of deriving the minimum distance between 
ISI channel sequences was described. The minimum distance for SF-MLSD and 
MFB-MLSD receivers was then calculated. As expected, the systems with MFB- 
MLSD receivers had a larger minimum distance relative to the corresponding SF- 
MLSD system. Also, the systems with square root raised-cosine filters had larger 
d mm than the corresponding systems with rectangular filters. The probability of 
error for the receivers was then obtained via Monte-Carlo computer simulation. 
First, it was verified that the performance obtained from computer simulations 
was below the upper bound calculated from d min . Next, the performance of the 
MLSD receivers was compared to systems with adaptive equalizers. The MLSD 
receivers provide a lower bound for the probability of error performance of the 
adaptive equalizers, and helped gauge the performance of the adaptive equalizers. 
Finally, the performance of the MFB-MLSD receivers for systems with square 
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root raised-cosine filters indicate that it is possible to obtain performance close to 
that of the AWGN. channel for non-constant modulus signals through nonlinear 
satellite channels. 

5.2 Suggestions for Further Research 

The results presented in this dissertation were derived from computer sim- 
ulation. In some cases, the performance of a particular receiver may justify a 
hardware implementation. In the case of the Volterra equalizers, issues such as 
precision and dynamic range would need to be studied before undertaking a hard- 
ware development. In the case of the MLSD receivers, a hardware implementation 
would require obtaining an accurate FSM model for an existing physical system. 
In such a case, further research involving modeling an actual nonlinear bandlim- 
ited satellite channel may be necessary. 

Several areas of research are possible as an extension to the equalizers of 
Chapter 3. Straightforward extensions to the adaptive Volterra equalizers are 
possible. For example, nonlinear decision feedback and ISI cancellers have not 
been studied for a satellite channel. However, these latter structures have been 
considered for voiceband and magnetic recording channels [48 - 50]. The perfor- 
mance results of the Volterra equalizer and FSE- Volterra equalizer can serve as a 
baseline for comparison of the performance of decision feedback and ISI canceller 
structures. Also, the MFB-MLSD receivers provide a lower bound on probability 
of error performance. 
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The MLSD receivers of Chapter 4 showed very good performance when used 
with rectangular and raised-cosine filtering. However, for the nonlinear satellite 
channel with memory, in order to implement an MLSD receiver a FSM model 
of the channel (or an equivalent channel description) is necessary. For an actual 
channel, obtaining such a description may not be possible. In such a case, an 
adaptive MLSD receiver as in [51,. 52] for linear channels may be useful. The 
structure and performance of these receivers for the nonlinear channel would need 
to be developed. 

The methods discussed in this dissertation have been focused on the receiver. 
However, methods which combine equalization and coding as in [53] have recently 
been studied for magnetic recording channels. These methods may also give sig- 
nificant probability of error improvement for the nonlinear satellite channel. Also, 
adaptive inverse modeling where the adaptive inverse filter is placed forward of 
plant (i.e., pre-equalization) as in the filtered-x LMS algorithm [54, ch. 11] may 
be useful for the nonlinear bandlimited satellite channel. 
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A. Computer Program Listings 


A.l. Volterra Equalizer Program 


/** *********************************************************** / 


/* pskvltre . c 

- A C-program which interfaces with MATLAB 

*/ 

/* 

for simulating a passband Volterra equalizer */ 

/* 

in baseband. MATLAB interfaces with this 

*/ 

/* 

program via the GATEWAY routine. The 

*/ 

/* 

GATEWAY routine then calls the computational 

*/ 

/* ‘ 

routine . 

*/ 


/*************************************************************/ 


#include <math.h> 

#include <stdio.h> 

#include " c : \mat lab\extern\include\mex . h " 

#define pi 3.141593 

/***♦ COMPUTATIONAL ROUTINE ***♦/ 
void pskvltre( 

Matrix *x, Matrix *wi, int K, 

Matrix *mssize, int M, int nltaps,char *aflag, 
Matrix **y, Matrix **wo, Matrix **d, Matrix **s) 

{ 


/*********************/ 
/* declare variables */ 
/ *********************/ 


double X,Xr ,Xi ,Xtemp; /* 
int carry; /* 
int wrow.wcol; /* 
int fig; /* 
int Mw; /* 
int i, j .k.l.dummy.NXjNeq.Ny; /* 
int nsymbol ; /* 
double *wPr,*wPi; /* 
double *dPr , *dPi ; /* 
double *xPr,*xPi; /* 
double *yPr , *yPi ; / * 
double *wiPr , *wiPi ; / * 
double *sPr; /* 


misc variables */ 

variable for calc, weight idx */ 

weight row and column var. */ 

flag all weights updated */ 

number of rows in weight matrix */ 

misc counters and max counts */ 

samples per symbol*/ 

output weight pointers */ 

decision pointers */ 

input pointer */ 

output pointer */ 

input weight pointer */ 

symbol pointer */ 
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double *mssizePr; 

double err_r,err_i,err; 

int *nidx; 

double ssize; 

int log2M; 

int xidx; 

int nlwflag; 

int hinltap; 

int lonltap; 

double *Xr_mat; 

double *Xi_mat ; 

int order; 

double alphar [16] , alphai [16] ; 
int betal [16] ,beta2[16] ; 
int beta3[16] ,beta4[16] ; 


/* step sizes pointer */ 

/* error variables */ 

/* nl term index array */ 

/* current step size */ 

/* log2(M) */ 

/* nl comb out matrix idx */ 

/* no. of non lin weights */ 

/* hi index of non lin weights */ 
/* lo index of non lin weights */ 
/* real NL comb output matrix */ 
/* imag NL comb output matrix */ 
/* current order variable */ 

/* symbol real and imag */ 

/* bit assignment variables */ 

/* bit assignment variables */ 


void pskdecnCint M, double yr, double yi, 

double *alphar , double *alphai, 

int *betal, int *beta2, int *beta3, int *beta4, 

double *dr, double *di , 

double *s) ; 


/*************************************/ 
/* Prepare for Decision Making */ 

/* define symbol and bit assignments */ 
/*************************************/ 


if (M==4) { 

alphar.[0]=l/sqrt (2.0) jalphai [0]=l/sqrt(2.0) ; /* data=ll */ 

alphar [l]=-l/sqrt(2.0) ;alphai[l]=l/sqrt(2.0) ; /* 01 */ 

alphar [2]=-l/sqrt(2.0) ;alphai[2]=-l/sqrt(2.0) ;/* 00 */ 

alphar [3] =l/sqrt (2.0) jalphai [3] =-l/sqrt (2.0) ; /* 10 */ 

. betal [0]=l;beta2[0]=l; 
betal [1] =0 ; beta2 [1] =1 ; 
betal [2] =0 ; beta2 [3] =0 ; 
betal [3] =1 ; beta2 [2] =0 ; 

> 

if (M==8) { 

alphar [0] =0 . 9239 ; alphai [0] =0 . 3827 ; 

- alphar [1] =0 . 3827 ; alphai [1] =0 . 9239 ; 
alphar [2] =-0 . 3827 ; alphai [2] =0 . 9239 ; 
alphar [3] =-0 . 9239 ; alphai [3] =0 . 3827 ; 
alphar [4] =-0 . 9239 ; alphai [4] =-0 . 3827 ; 
alphar [5] =-0 . 3827 ; alphai [5] =-0 . 9239 ; 
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alphar [6] =0 . 3827 ; alphai [6] =-0 . 9239 ; 
alphar [7] =0 . 9239 ; alphai [7] =-0 . 3827 ; 
betal [0] =0 ; beta2 [0] =0 ; beta3 [0] =0 ; 
betal [1] =0 ; beta2 [1] =0 ;beta3 [1] =1 ; 
betal [2] =0 ; beta2 [2] =1 ;beta3 [2] =1 ; 
betal [3] =0 ; beta2 [3] =1 ; beta3 [3] =0 ; 
betal [4] =1 ; beta2 [4] =1 ;beta3 [4] =0 ; 
betal [5] =1 ; beta2 [5] =1 ;beta3 [5] =1 ; 
betal [6] =1 ; beta2 [6] =0 ; beta3 [6] =1 ; 
betal [7] =1 ; beta2 [7] =0 ; beta3 [7] =0 ; 

> 

if (M==16) { 

alphar [0] =0 . 9808 ; alphai [0] =0.1951; 
alphar [1] =0 . 8315 ; alphai [1] =0 . 5556 ; 
alphar [2] =0 . 5556 ; alphai [2] =0 . 8315 ; 
alphar [3] =0 . 1951 ; alphai [3] =0 . 9808 ; 
alphar [4] =-0 . 1951 ; alphai [4] =0 . 9808 ; 
alphar [5] =-0 . 5556 ; alphai [5] =0 . 8315 ; 
alphair [6] =-0 . 8315 ; alphai [6] =0 . 5556 ; 
alphar [7] =-0 . 9808 ; alphai [7] =0 . 1951 ; 
alphar [8] =-0 . 9808 ; alphai [8] =-0 . 1951 ; 
alphar [9] =-0 . 8315 ; alphai [9] =-0 . 5556 ; 
alphar [10] =-0 . 5556 ; alphai [10] =-0 . 8315 ; 
alphar [11] =-0 . 1951 ; alphai [11] =-0 . 9808 ; 
alphar [12] =0 . 1951 ; alphai [12] =-0 . 9808 ; 
alphar [13] =0 . 5556 ; alphai [13] =-0 . 8315 ; 
alphar [14] =0 . 8315 ; alphai [14] =-0 . 5556 ; 
alphar [15] =0 . 9808; alphai [15] =-0 . 1951 ; 

betal [0] =0 ; bet a2 [0] =0 ; beta3 [0] =0 ; bet a4 [0] =0 ; 
betal [l]=l;betaL2[l]=0;beta3[l]=0;beta4[l]=0; 
betal [2] =1 ; bet a2 [2] =1 ; beta3 [2] =0 ; beta4 [2] =0 ; 
betal [3] =0 ; bet a2 [3] =1 ; bet a3 [3] =0 ; beta4 [3] =0 ; 
betal [4] =0; beta2 [4] =1 ; beta3 [4] =0 ; beta4 [4] =1 ; 
betal [5] =1 ; beta2 [5] =1 ; beta3 [5] =0 ; beta4 [5] =1 ; 
betal [6] =1 ; beta2 [6] =0 ; beta3 [6] =0 ; beta4 [6] =1 ; 
betal [7] =0 ; betai2 [7] =0 ; beta3 [7] =0 ; beta4 [7] =1 ; 
betal [8] =0 ; beta2 [8] =0 ;beta3 [8] =1 ; beta4 [8] =1 ; 
betal [9] =1 ; beta2 [9] =0 ; beta3 [9] =1 ; bet a4 [9] =1 ; 
—betal [10] =1 ;beta2 [10] =1 ;beta3 [10] =1 ;beta4 [10] =1 
betal [11] =0 ; beta2 [1 1] =1 ;beta3 [11] =1 ; beta4 [11] =1 
betal [12] =0 ; beta2 [12] =1 ; beta3 [12] =1 ;beta4 [12] =0 
betal [13] =1 ;beta2 [13] =1 ;beta3 [13] *1 ;beta4 [13] =0 
betal [14] =1 ; bet a2 [14] =0 ; beta3 [14] =1 ; beta4 [14] =0 
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betal [15] =0 ;beta2 [15] =0 ; beta3 [15] =1 ;beta4 [15] =0; 


> 


/***♦**♦*/ 


/* MISC */ 

/********/ 

mssizePr=mxGetPr(mssize) ; /* step size vec pointer */ 

/* allocate nl comb */ 

/* real/imag matrics */ 

Xr_mat= (double *)mxCalloc(Mw*Neq*sizeof (double)) ; 

Xi_mat= (double *)mxCalloc(Mw*Neq*sizeof (double)) ; 
nidx=mxCalloc(2*K,sizeof (int)) ; /* allocate weight idx array*/ 

Weq=mxGetN(wi) ; /* weight matrix col's */ 

Mw=mxGetM(wi) ; /* weight matrix rows */ 

Nx=mxGetN (x) ; / * input length */ 

Ny=Nx+Neq-l; /* output length */ 

log2M=log((double)M)/log((double)2) ; /* log2(M) */ 

hinltap=(Neq-l)/2 + (nltaps-l)/2; /* hi nonl tap */ 

lonltap=(Neq-l)/2 - (nltaps-l)/2; /* lo nonl tap */ 


/*****************************/ 
/* Initialize Output Weights */ 
/*****************************/ 


*wo=mxCreateFull (Mw,Neq, COMPLEX) ;/* allocate output weights */ 


wiPr=mxGetPr(wi) ; 
wiPi=mxGetPi(pi) ; 
wPr=mxGetPr(*wo) ; 
wPi=mxGetPi (*wo) ; 
for (i=0;i<Neq;i++) 
for (j=0; j<Mw; j++) { 
wPr [i*Mw+j]=wiPr[i*Mw+j] ; 
wPi[i*Mw+j]=wiPi[i*Mw+j] ; 
» 


/* real in weight pointer */ 
/* imag in weight pointer */ 
/* real out weight pointer */ 
/* imag out weight pointer */ 

/* init out weights */ 

/* = in weights 


/****************************************/ 
/* Equalizer Output and Weight Update */ 

/*** *************************************/ 


*y=mxCr eateFull ( 1 , Ny , COMPLEX) ; 

*d=mxCreateFull(l ,Ny, COMPLEX) ; 

*s=mxCreateFull(log2M,Ny,REAL) ; 
yPr=mxGetPr(*y) ; 


/* create out vec */ 
/* create decision vec */ 
/* create symbol vec */ 
/* real out vec pointer*/ 
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yPi=mxGetPi (*y) 
dPr=mxGetPr ( *d) 
dPi=mxGetPi (*d) 
sPr=mxGetPr ( * s ) 
xPr=mxGetPr(x) ; 
xPi=mxGetPi(x) ; 


for (i=Neq;i<=Nx;i++) { 
yPr [i-l]=0; yPi[i-l]=0; 
wrow=0;wcol=0; 
f or ( j =1 ; j <=2*K-1 ; j ++) 
nidx [j]=0; 
for (k=l;k<=K;k++){ 
flg=0; 
xidx=0; 
while(lflg) { 

Xr=l . 0 ; Xi=0 ; nlwf lag=l ; 
if (k>l){ 

for (1=1 ; l<=2*k-l ; 1++) 
if (nidx [2*k-l-l]<lonl tap II nidx[2*k-l-l]>hinltap) 
nlwf lag=0 ; > 

if (nlvflag==l) ■( /* Is this NL term? */ 

for(j=l; j<=2*k-l; j++) { /* Combine NL Terms */ 

if(j>=k+l) { 

Xtemp=Xr*xPr [i-nidx [2*k-l-j] -1] 

+Xi*xPi [i-nidx [2*k-l-j]-l] ; 

. Xi=-Xr*xPi [i-nidx [2*k-l-j]-l] 

+Xi*xPr [i-nidx [2*k-l-j]-l] ; 

Xr=Xtemp;> 
else { 

Xtemp=Xr*xPr [i-nidx [2*k-l- j] -1] 

-Xi*xPi [i-nidx [2*k-l-j]-l] ; 

Xi=Xr*xPi [i-nidx [2*k-l-j]-l] 

+Xi*xPr [i-nidx [2*k-l-j]-l] ; 

Xr=Xtemp ; }} 

Xr _mat [xidx] =Xr ; /* Save NL combiner output */ 

Xi_mat [xidx] =Xi ; 
xidx++ ; 

/* Accumulate Output */ 

yPr [i-1] +=Xr*wPr [wrow+wcol*Mw] -Xi*wPi [wrow+wcol*Mw] ; 
yPi [i-1] +=Xi*wPr [wrow+wcol*Mw] +Xr*vPi [wrow+wcol*Mw] ; } 
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/* imag out vec pointer*/ 
/* real decision pointer*/ 
/* imag decision pointer*/ 
/* symbol pointer */ 

/* real in vec pointer*/ 

/* imag in vec pointer*/ 

/* EQ Fully Loaded */ 

/* init i-th out to zero */ 

/* init weight matrix idx’s*/ 
/* init weight idx to zero */ 
/* init nl term idx array */ 


/* for all lin and nl weights*/ 



/* update nl term idx */ 

/* update weight matrix idx’s */ 
if (wool < Neq-1) wcol++; 
else {wcol=0; wrow++;} 
carry=l ; 

for (l=0;l<2*k-l ;1++) { /*update nidx[] modulo Neq*/ 
nidx [1] +=carry ; 

if (nidx[l] <Neq) carry=0; 
else { if (nidx [2*k-2] ==Neq) 

flg=l; nidx[l]=0; carry=l;> 

} /* end of for 1 */ 

} /* end of while !flg */ 

} /* end of for k */ 

/* make decision */ 

pskdecn(M,yPr [i-1] ,yPi[i-l] ,alphar,alphai,betal,beta2, 
beta3,beta4,&dPr [i— 1] ,&dPi [i-1] ,&sPr [log2M*(i-l)] ) ; 

/ ****** ***** ********/ 

/* WEIGHT ADAPTION */ 
/*******************/ 
if (!strcmp(aflag,"on")) { 
err_r=dPr [i-l]-yPr [i-1] ; 
err_i=dPi [i-1] -yPi [i-1] ; 
wcol=0 ; wrow=0 ; xidx=0 ; 
for (k=l ;k<=K;k++){ 
order=2*k-l; 

ssize=mssizePr [k-1] ; /* Step Size For This Order */ 

for(j=0; j<(int)pow((double)Neq, (double) order) ; j++){ 
Xr=Xr_mat [xidx] ; 

Xi=Xi_mat [xidx] ; 

wPr [wrow+wcol*Mw]+=ssize*(Xr*err_r+Xi*err_i) ; 
wPi [wrow+wcol*Mw] +=ssize* (Xr*err_i-Xi*err_r) ; 
wcol++ ; 

if (wcol==Neq) {wrow++; wcol=0;> 
xidx++ ; }} 


}/* end if ! strcmp(af lag, "on") */ 
}/* end for i */ 

} /** END COMPUTATION ROUTINE **/ 
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/*******************************/ 
/***♦ GATEWAY routine **********/ 
/*******************************/ 
void mexFunction( 

int nlhs , Matrix *plhs [] , 
int nrhs. Matrix *prhs[]) 

{ 

int Neq.flg; 
int M; 

int i , j , xcomplx , wcomplx ; 

"double etrgt; 
int mcycle; 
char adpf lagi [4] ; 
int' dummy; 
int nx,nw; 
int m; 
int k; 
int nltaps; 

Matrix *x , *win , *nsymbol , *af lg , *mssize ; 
double *apPr ; 
char aflag[10] ; 

Matrix *ssizes ; 


/* modulation order */ 

/* input signal and weight flags */ 

/* error target */ 

/* maximum number of training cycles*/ 
/* adaption flag */ 

/* dummy variable */ 

/* columns for input and weights */ 

/* number of rows for weights */ 

/* order of non-linearity is 2k-l*/ 


/*** PROCESS I/O ARGUMENTS ***/ 

if (nrhs !=6) mexErrMsgTxt ("6 input arguments are required\n") ; 
else 

if (nlhs>4) mexErrMsgTxt ( "maximum of 4 output arguements\n") ; 
if (CmxIsNumeric(prhs[0])==0) I I (mxGetM(prhs [0] ) !=1)) 
mexErrMsgTxt ("input signal must be an 1 x n numeric vector\n"); 
if ( mxIsNumeric(prhs[l] )==0 ) 

mexErrMsgTxt ( "weights must be an m x n numeric vector\n“); 
m=mxGetM (prhs [1] ) ; 

Neq=mxGetN (prhs [1] ) ; 
i=3;j=l; k=0; flg=0; 
while (!f lg) { 
k++; 

if (m==j) break; 

-j+=(int)pow( (double)Neq, (double) (i— 1) ) ; 
i+=2; 
if (m<j) 

{ mexErrMsgTxt ("cannot derive, k from input weights\n") ;> 

> 


114 



> 


if ((mxIsNumeric(prhs [2] )==0) II (mxGetM(prhs [2] ) != 1) 

II (mxGetN (prhs [2] ) !=1)){ 

mexErrMsgTxt ("modulation order, M, must be a real scaler\n");> 
if ( (*mxGetPr(prhs [2] ) !=2) kk (*mxGetPr(prhs [2] ) ! =4) 

kk (*mxGetPr(prhs [2] ) !=8)&& (*mxGetPr (prhs [2] ) ! =16) ) 
mexErrMsgTxt ("modulation order, M, must be 2, 4, 8, or 16\n") ; 
if ( mxGetN (prhs [3] ) ! =k) 

mexErrMsgTxt ( "number of step sizes must agree with order\n"); 
if (mxIsNumeric(prhs[4])==0 II mxGetN (prhs [4] ) !=1 
I I mxGetM(prhs [4] ) !=1) 

mexErrMsgTxt ("No. of nltaps must be a numeric scaler\n"); 
nltaps=*mxGetPr(prhs [4] ) ; 
if (nltaps7,2!=l) 

mexErrMsgTxt ("No. of Nonlinear taps must be odd\n") ; 
if (mxIsString(prhs [5] )==0) 

mexErrMsgTxt ("adaption flag must be ascii\n"); 

/* CREATE OUTPUT VARIABLES and CALL COMP ROUTINE*/ 
x=prhs [0] ; 
win=prhs [1] ; 

M=*mxGetPr (prhs [2] ) ; 
ssizes=prhs [3] ; 

mxGetString(prhs [5] ,aflag, 10) ; 
pskvltre(x, win, k,ssizes,M, nltaps, aflag,&plhs [0] , 
feplhs [1] ,&plhs [2] .feplhs [3 ] ) ; 


void pskdecn(int M, double yr, double yi, 
double *alphar, double *alphai, 
int *betal, int *beta2, int *beta3,int *beta4, 
double *dr, double *di, 
double *s) { 
double yangle; 
int k; 
if (M==2) { 

if (yr > 0 ) {*dr=l;*s=l;> 
else {*dr=-l;*s=0;} 

> 

else { 

—if (yr==0 kk yi==0) yangle=0; 
else yangle=atan2(yi ,yr) ; 
if (yangle<0) yangle+=2*pi ; 
k=floor (yangle/ (2*pi/M)) ; 
if (M==4) { 
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*dr=alphax [k] ; 
*di=alphai [k] ; 
s [0] =betal [k] ; 
s[l]=beta2[k] ; 
> 

if (M=8) { 
*dr=alphax [k] ; 
*di=alphai [k] ; 
s [0] =betal [k] ; 
s [l]=beta2[k] ; 
s [2] =beta3 [k] ; 
> 

if (M== 16 ) { 
*dr=alphar [k] ; 
*di=alphai [k] ; 

' s [0] =betal [k] ; 
s [l]=betcL2[k] ; 
s [2] =beta3 [k] ; 
s [3] =beta4 [k] ; 
> 

> 

> /* end pskdecn */ 



A. 2. State Table Generation Program 
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srcfb: a MATLAB program for Generating 

State Table 

Matched Filter Bank Rx 
Nonlinear Satellite Channel 
CHANNEL PARAMETERS 

- Tx = square root raised cosine 

- Rx = Matched Filter Bank 

- TWT 0 dB backoff 
INSTRUCTIONS 

- Change the System Parameters 

- Indicate the file in which to store the state machine 

- Execute from within MATLAB 
COMM. SYS. COMPONENTS 

Transmitter - square root raised cosine 

Channel - TWT 

Receiver - matched filter bank 

STATE MACHINE FILE FORMAT 

The file will have M“(L+1) entries (i.e., lines). The first 
field of a line contains the input. The input is an integer 
in the range 0, 1, 2, ... M-l. The next set of fields 
contain the L previous inputs which correspond to the state . 
The next field contains the channel output (note the output 
may be complex). The number of outputs per line is 2*nsym, 
where the nsym parameter, set below, indicates the number of 
samples per symbol. The factor of 2 is because the outputs 
cure complex. 

The following is a listing of the first few lines for 
a system with modulation order M=4 memory L=2 and 1 
sample per symbol. 

000 -0.705858 -0.705858i 
100 -0.757653 -0.705858i 
- 200 -0.705858 -0.757653i 

300 -0.757653 -0.757653i 
010 0.51781 -0.705858i 

110 0.466016 -0.705858i 

210 0.51781 -0.757653i 
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y. 

% 

7 . 

y. 

% 

•/. 

y. 

•/. 

*/. 

*/. 

y. 

y. 

% 

y. 

•/. 

y. 

y. 

•/. 

y. 

*/. 

% 

% 

% 

% 


3 1 0 0.466016 -0.7576531 

0 2 0 -0.705858 0.5178H 

1 2 0 -0.757653 0.5178H 

OTHER MATLAB PROGRAMS CALLED BY THIS PROGRAM 

These axe straightforward and are only summarized here 

randbits - generate random bits 

pskmd - bit to symbol mapping (i.e., baseband modulate) 

rcff - raised cosine filter, square root raised cosine 

with appropriate flag settings. Used here as 
square root raised cosine filter, 
xltwt - low-pass equivalent TWT model, implementation 
of Saleh’s analytical model (IEEE Trans. Comm, 
vol . COM-29. No. 11, pp. 1715-1720, Nov. 1981) 
symsync - find sample number which gives largest 

correlation between two complex vectors. This 
is used to synchronize the received signal with 
the input signal. This way, a comparison of bit 
errors may be made. This also generates the 
phase offset between the two signals, 
eavg - calculate the average energy in a signal given 
the number of samples per symbol. 
dec2bin - convert a decimal number to a binary number 


'/,%% SYSTEM Parameters 


Rsym=l ; 
nsym=4 ; 
fs=nsym*Rsym; 
msm_flg=’on’ ; 
wsm_flg=’on’ ; 


'/. Sample Rate 
% Samples Per Symbol 
'/. Sample Rate 
*/, make state machine flag 
'/, write state machine flag 


Lf=l; ’/. Comm System Memory future 
Lp=l; '/, Comm System Memory past 
L=Lf+Lp; */, Total Memory 
M=8; 7 . Modulation Order 
K=log(M)/log(2) ; % log2(M) 
beta=l; 7 , Rolloff parameter 

twtp=-£l .9638 0.9945 2.5293 2.8168]; 7 . TWT Parameters 
backoff=0; 7 * TWT backoff 

Nout=M~(L+l)-l ; y, no. of outputs <=M“(L+1)-1 
*/, M"(L+1)-1 for entire state 

7 , machine 


118 



mu 

% FILENAME.' »% 

•/x/x/x/x/x/.m 


if msm_f lg==’on J 
if wsm_f lg== , on > 

f id=fopen( , frcl_811 , , ’w’) ; '/. indicate state table file here 

end 
end 


m 


tx and rx long sequence */,'/,•/, 
YXY. good for eye and scatter YXl 
Y.YX also determine sample offset 


clear bl xldlta xltx xlprf % clr variables avoid confusion 
clear xltwt xlrx y xlpof 

bl=randbits(K,512) ; ’/, random bit sequence 

bl= [zeros (K, 50) bl zeros (K, 50) ] ; ’/, new bit sequence pad with zeros 

xldlta=pskmd(bl,nsym) ; */. tx impulse sequence 

xltx=rcff (xldlta, nsym.nsym* 17, beta, 1,0,0) ;Xraised cosine tx filter 

xltwt=ltwt (xltx, twtp, backoff ,0) ; */, TWT 

e_avg=eavg(xltwt(10:length(xltwt)-10) ,nsym) ; '/# avg sym. energy 
xltwt=xltwt/sqrt(e_avg) ; '/.normalize 

’/, correlate with tx seq 

[xx smpofst phsof st]=symsync (xldlta, xltwt ,nsym, 100) ; 
xltwt=xltwt*exp(-sqrt (-l)*phsofst) ; '/, phase sync 
of st=smpof st+(50+Lp)*nsym; '/. sample offset 


mfflmmmxrafflmmfflii 

Create State Machine Model 


*/.’/X/. write oversampled ch. output */X/« 

YXJX to file V/X 


clear in_state b bits xdlta xtx */, clr variables 

clear xprf xtwt xpof xrx rxo rxs 

if msm_f lg== , on’ 

eav=0; 

erx=0 ; 

in_state=zeros(l,l+L) ; */. initialize variables 

b=zeros(K, 1+L) ; 

for(i=0:Nout) 
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f or (k=l :L+1) 

b( : jk)=dec2bin(in_state(k) ,K) ’ ; 7 . input and state sequence 
end 

bits=[randbits(K,50) b randbits(K,50)] ; '/, tx sequence 
xdlta=pskmd(bits,nsym) ; '/.modulate 

xtx=rcff(xdlta,nsym,nsym*17, beta, 1,0,0) ;'/.sqr. root rais. cos. 
xtwt=ltwt (xtx , twtp , backoff , 0) ; '/, TWT 

xtwt=xtwt/sqrt (e_avg) ; 7, normalize energy 

xtwt=xtwt*exp(-sqrt (-l)*phsofst) ; 7. phase sync 

h=xtwt(ofst :of st+nsym-1) ; 7. channel output vec 

7 . WRITE TO FILE 7 . 

for(j=l:L+l) 

fprintf (f id, ’’/.g ’ ,in_state(j)) ; 7. write input.state 
end 


if (M>2) 

' for j=l:nsym 


7. complex 


7. write ch. output 

fprintf (fid, * */.g 7.gi’ ,real(h(j)) ,imag(h(j))) ; 
end 

else 7. real 

for j=l:nsym 

fprintf (fid, ,real(h(j))) ; 7. write ch. out 


end 

end 

fprintf (fid, ’\n’) ; 7. done writing this 

7. line 

carry=l ; 7. determine next 

for(k=l:L+l) 7 . input state sequence 

if (carry==l) in_state(k)=in_state(k)+carry; ,end 
if (in_state(k)==M) in_state(k)=0; carry=l ; 
else carry=0;, end 
end 


fprintf ( ,# /.d ’,i); 7. some output to let 

if (rem(i+l,10) == 0) fprintf ( ’ \n ' ) ; , end'/, user know the prog. 

7. is doing something 

end 

if wsm_f lg= =, on’ 
f close(f id) ; 
end 

end 
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A. 3. Matched Filter Bank Receiver Program 


/*************************************************************/ 


/* cv2.c: A program for simulating the performance */ 
/* of a matched filter bank receiver followed */ 
/* by a Viterbi Detector. The program first */ 
/* reads in the state table description and */ 
/* " uses this to generate channel outputs and */ 
/* to derive Viterbi detector path metrics. */ 
/* The program assumes the state table */ 
/'* contains the oversampled outputs from */ 
/* the channel. */ 


/*************************************************************/ 
#include <stdio.h> 

#include <math.h> 

#include <stdlib.h> 


/ ******* ****** *************/ 
/*** TYPE decelerations ***♦/ 
/**************************/ 

typedef struct{ 
double re; 
double im; 

}cmplx_dbl ; 
typedef struct{ 
float re; 
float im; 

}cmplx_f It ; 
typedef struct{ 
int * input ; 

int * state; 

cmplx_flt ^output; 

} stable; 
typedef struct{ 
int * input; 
int *state ; 

}tmsmt ; 
typedef struct{ 
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int path_lgth; 
int *prv_stbl_idx; 
double *met_new; 
double *met_old; 
int *pmem ; 
int *pmem_old; 
double *fb; 
double *fbsig; 

>trellis ; 

/ **************/ 

/*** MAIN ***/ 
/**************/ 

int mainO { 

/* declare variables */ 


FILE *st_tbl_file; 

/* 

FILE *st_tbl_f ile_tx 

; /* 

int num_states; 

/* 

int num_states_tx ; 

/* 

int M; 

/* 

int L,L_tx; 

/* 

int nsym; 

/* 

float n; 

/* 

float MaxSymbols; 

/* 

float MaxErrs; 

I* 

float EsNodb; 

/* 

float EsAvg; 

/* 

stable stbl; 

/* 

stable stbl_tx; 

/* 

trnsmt tx; 

/* 

trellis trls; 

/* 

int PathLgth; 

/* 

cmplx_dbl noise; 

/* 

cmplx.dbl *ch_output; /* 

int tx_out_idx; 

/* 

int i,j,k; 

/* 

int sym_err_cntr ; 

/* 

int out; 

/* 

int-ref ; 

/* 

int *ref_reg; 

/* 

int cntrl=0; 

/* 


state table file for receiver */ 
state table file for transmitter */ 
number of states in receiver */ 
number of states for transmitter */ 
modulation order */ 
memory length */ 
samples per symbol */ 
symbol number */ 

maximum number of tx’d symbols */ 

maximum number of errs */ 

symbol to noise spec dens ratio */ 

Average Signal Energy */ 

state table for receiver */ 

state table for transmitter */ 

transmitter structure */ 

trellis structure */ 

path memory length*/ 

noise variable */ 

ch ann el output */ 

ch ann el output index */ 

misc ints */ 

error counter */ 

detector output */ 

noiseless tx output */ 

ref delay register-for decoding delay*/ 

Number of symbols counter */ 
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/* declare functions */ 

void load_st_tbl_cmplx(FILE *st_tbl_file, 

stable stbl , int M,int L,int nsym) ; 
void print_st_tbl_cmplx(stable stbl, int M,int L,int nsym); 
void updt_tx(tmsmt tx.int M,int L) ; 
void updt_fb (stable stbl, cmplx_dbl *ch_output, 

trellis trls, int M, int L, int nsym); 
void updt.trls (stable stbl, trellis trls, int M, int L) ; 
int get_stbl_idx(trasmt tx, int M, int L) ; 
int get_trls_out (trellis trls, int M,int L) ; 
void add.awgn (double esavg, double esnodb, 

cmplx_dbl *ch_ output , int nsym); 
char st_tbl_f ilename [100] ; 
char st_tbl_f ilename _tx [100] ; 
void get_ch_out (stable stbl_tx,int tx_out_idx, 
cmplx_dbl *ch_output , int nsym) ; 
void init_fbsig(stable stbl, trellis trls, int M, 
int L , int nsym) ; 

/* get user input */ 

pr int f ( "Enter MaxSymbols, MaxErrs, Es/No(dB), EsAvg: "); 

scanf("7,f */,f */,f */,f " , feMaxSymbols , feMaxErrs , feEsNodb , &EsAvg) ; 

pr int f ("Enter Symbol Set Size, Samples/Symbol, Tx Mem,Rx Mem: ") 

scanf ("7«d 7.d */,d 7,d" ,&M,&nsym,&L_tx,&L) ; 

printf ( "Enter Viterbi Detector Path Memory Length: "); 

scanf ( "7.d" , fePathLgth) ; 

printf ( "Enter Tx State Table Filename: "); 
scanf ("7.s" ,st_tbl_f ilename _tx) ; 
printf ("Enter Rx State Table Filename: "); 
scanf ("7.s" ,st_tbl_f ilename) ; 


if ((st_tbl_file=fopen(st_tbl_f ilename, "r"))==NULL) { 
printf ("ERROR: could not open \"7.s\" for reading\n" , 
st _tbl_f ilename) ; 
exit (1) ; 

> 

if (-(st_tbl_file_tx=fopen(st_tbl_filename_tx,"r"))==NULL) { 
printf ("ERROR: could not open \"7.s\" for reading\n" , 
st_tbl_f ilename_tx) ; 
exit(l) ; 

> 
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/* memory allocation */ 
num_states=pow((double)M, (double)L) ; 
num_states_tx=pow( (double) M, (double)L_tx) ; 
stbl.input=(int *)malloc(M*num_states*sizeof (int)) ; 
stbl . state=(int *)malloc (M*L*num_states*sizeof (int ) ) ; 
stbl . output=(cmplx_f It *)malloc (nsym*M*num_states* 

sizeof (cmplx.f It) ) ; 

stbl.tx. input =( int *)malloc(M*num_states_tx*sizeof (int)) ; 
stbl_tx.state=(int *)malloc(M*L_tx*num_states_tx*sizeof (int)) ; 
stbl_tx.output=(cmplx_flt *)malloc(nsym*M*num_states_tx* 

sizeof (cmplx_f It)) ; 

tx.state=(int *)malloc (L_tx* sizeof (int)) ; 
tx.input=(int *)malloc (sizeof (int)) ; 

' trls . met _new= (double *)malloc(num_states*sizeof (double)) ; 
trls.met_old= (double *)malloc(num_states*sizeof (double)) ; 
trls.pmem=(int *)malloc(num_states*PathLgth*sizeof (int)) ; 
trls.pmem_old=(int *)malloc(num_states*PathLgth*sizeof (int)) ; 
trls.prv.stbl.idx^dnt *)malloc(num_states*sizeof (int)) ; 
trls .fb= (double *)malloc(M*num_states*sizeof (double)) ; 
trls .fbsig® (double *)malloc(M*num_states*sizeof (double) ) ; 
ch_output=(cmplx_dbl *)malloc(nsym*sizeof (cmplx_dbl) ) ; 
ref_reg=(int *)malloc(PathLgth*sizeof (int)) ; 


/* load rx and tx state table */ 

load_st_tbl_cmplx(st_tbl_f ile.stbl ,M,L,nsym) ; 
load_st_tbl_cmplx(st_tbl_f ile_tx , stbl.tx ,M , L_tx , nsym) ; 


/* initialize variables */ 

for(i=0;i<L_tx; i++) tx. state [i]=0; 
*tx.input=0; 

for(i=0;i< num_states*PathLgth;i++) 
trls.pmem[i]=0; 

for(i=0;i< num_states*PathLgth ; i++) 

-t rls . pmem_old [i] =0 ; 
for(i=0;i< PathLgth ; i++) ref_reg[i]=0; 

♦trls . met_new=0 ; 

♦trls .met_old=0; 

trls . path_lgth=PathLgth ; 
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init_fbsig(stbl,trls,M,L,nsym) ; 

/* simulate the tx, channel, MLSE rx */ 

sym_err_cntr=0 ; 
n=0; 

while (n<MaxSymbols && sym_err_cntr<MaxErrs) { 

/* use state table to */ 

updt_tx(tx,M,L) ; /* generate state table */ 

tx_out_idx=get_stbl_idx(tx,M,L) ; /* address for the next */ 

/* output */ 


get_ch_out (stbl_tx,tx_out_idx, /* get output from */ 

ch.output ,nsym) ; /* state table and */ 

add_awgn(EsAvg,EsNodb,ch_output ,nsym) ;/* add noise */ 


updt_fb(stbl,ch_output ,trls,M,L,nsym) ;/* update filter bank */ 


/* update trellis */ 

updt_trls(stbl,trls,M,L) ; /* including path met. */ 

/* get output with */ 

out=get_trls_out(trls,M,L) ; /* best metrics */ 

for(i=trls.path_lgth-l;i>0;i — ) /* delay inputs */ 

ref_reg[i]=ref_reg[i-l] ; /* so can compare */ 

ref _reg[0]=*tx. input; /* with outputs */ 

ref=ref _reg[trls.path_lgth-l] ; 


if (out!=ref) { /* print and update */ 

sym_err_cntr++ ; /* stats on errors */ 

printf ("Symbol Error, n =, /.g, Detect=*/.d, 

Ref=7.d SymErrs=y.d SER='/.f \n" , 
n+1 , out , ref , sym_err_cntr , 
sym_err_cntr/ (n+1) ) ; 

> 

if (++cntrl==10000){ /* print stats every */ 

printf ("n=y,g, S 5 nnErrs =, /,d, SER=y,f\n" ,n+l,/* 10K symbols */ 
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sym_err_cntr , sym_err _cntr/ (n+1) ) ; 
cntrl=0; 

> 

n++; 

> 

/* print final stats */ 

printf ("n=7.g, SymF.rr s=*/,d , SER=7.f\n", 

n, sym_err_cntr , sym_err_cntr/n) ; 

return(O) ; 

> 

/*** END of MAIN ***/ 

void load_st_tbl_cmplx(FILE *st_tbl_f ile, stable s’tbl, 

int M,int L,int nsym) { 

int num_states; 
int i , j ; 

num_states=pow( (double) M, (double)L) ; 
for(i=0;i< M*num_states ; i++) { 

f scanf (st_tbl_f ile , "7.d" ,&stbl . input [i] ) ; 

for(j=0;j< L; j++) fscanf (st_tbl_f ile, "%d",&stbl. state [j+i*L]) ; 
f or ( j =0 ; j <nsym ; j ++) 

f scanf (st_tbl_f ile, "7.f 7,fi", 

festbl. output [i*nsym+j] . re, &stbl. output [i*nsym+j] . im) ; 

> 

> 

void print_st_tbl_cmplx (stable stbl, int M,int L, int nsym) { 
int num.states; 
int i , j ; 

num_states=pow( (double) M, (double) L) ; 
for(i=0;i<M*num_states;i++) { 
printf ("7,d ", stbl . input [i] ) ; 

for(j=0; j<L; j++) printf ("7.d " ,stbl.state.[j+i*L] ) ; 
for(i=0; i<nsym; i++) 
printf (" 7.g 7.gi", 

stbl. output [i*nsym+j] . r e, stbl. out put [i*nsym+j] .im) ; 
printf ("\n"); 

> 

} - 

void updt_tx(trnsmt tx,int M,int L) { 
int i; 

for(i=L-l;i>0;i — ) tx. state [i]=tx. state [i-1] ; 
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tx . state [0] =*tx . input ; 
*tx . input=rand ( ) %M ; 


int get_stbl_idx(trnsmt tx, int M, int L) { 
int i ; 
int stidx; 
stidx=*tx . input ; 
for (i=0;i<L;i++) stidx+= 

tx. state [i] *pow( (double) M, (double) (i+1) ) ; 

return (stidx) ; 

> ‘ 

int get _snum (stable stbl, int stbl_idx,int M, int L) { 
int i ; 
int snum=0; 
for (i=0;i<L;i++) 

snum+=stbl. state [i+stbl_idx*L] *pow( (double) M, (double) (i)) ; 
return (snum) ; 

> 

int get _trls_out (trellis trls, int M,int L){ 
int num_states; 
int i ,best_state; 
double tmp; 

num_states=pow( (double) M, (double) L) ; 

best_state=0; 

f or ( i=0 ; i<num_states ; i++) 

if (trls .met_new[i] <trls.met_new[best_state] ) best_state=i 
if (trls. met _new[best_state]>le9) { 
tmp=trls .met_new[best_state ] ; 
f or(i=0; i<num_states; i++){ 
trls . met_new [i] -=tmp ; 
trls . met_old [i] -=tmp ; 

} 

> 

return (trls . pmem [ (best_state+l ) *trls . patb_lgth-l] ) ; 

> 


void -updt_trls (stable stbl, trellis trls, int M, int L) { 
int num_ states; 
int state; 

int acs(int state, stable stbl, trellis trls, int M, int L) ; 
void reg_xchg( stable stbl, trellis trls, int M, int L) ; 
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num_states=pow( (double) M, (double)L) ; 
for (state=0;state<num_states;state++) 

trls . prv_stbl_idx [state] =acs (state , stbl , trls , M , L) ; 
for (state=0 ; state<num_states ; state++) 

trls . met_old [state] =trls .met.new [state] ; 
reg_xchg (stbl , t rls , M , L) ; 

> 

int acs(int state, stable stbl, trellis trls, int M, int L) { 
int num.states; 
int i; 

int pvst_idx,best_pvst_idx; 
int prv.snum; 
double br .metric; 
double tmp.metric; 

int get_snum(stable stbl, int stbl_idx,int M, int L) ; 
for(i=0;i<M;i++) { 

pvst_idx=state+i*pow( (double)M, (double) (L)) ; 
br_metric=-2*trls . f b [pvst.idx] +trls . f bsig [pvst.idx] ; 
prv_snum=get_snum ( stbl , pvst _idx , M , L) ; 
tmp.metri c=br_metric+trls . met _old [prv.snum] ; 
if (i==0 ) { 

trls .met.new [state] =tmp_metric ; 
best_pvst_idx=pvst_idx ; 

> 

else if (tmp_metric<trls. met _new [state]) { 
trls . met.new [state] =tmp_metric ; 
best_pvst_idx=pvst_idx ; 

> . 

> 

return (best _pvst_idx) ; 

> 

void reg.xchg (stable stbl, trellis trls, int M, int L){ 
int i, state, num.states; 
int prv.state; 

int get.snum (stable stbl, int stbl_idx,int M, int L) ; 
num_states=pow( (double)M, (double)L) ; 

for(state=0;state<num_states;state++) { /*** REGISTER XCHG **♦*/ 
prv_state=get_snum(stbl , trls .prv_stbl_idx [state] ,M,L) ; 
f or ( i=0 ; i<trls . path.lgth ; i++) 

trls . pmem [i+st ate*trls . path.lgth] = 

trls .pmem_old[i+prv_state*trls .patb.lgtb] ; 

> 
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for(state=0;state<num_states;state++) { /*** REGISTER SHFT ****/ 

prv_state=get_snum(stbl ,trls .prv_stbl_idx[state] ,M,L) ; 
f or (i=trls .path_lgth-l ; i>0; i — ) 

trls .pmem[i+state*trls .path_lgth] = 

trls . pmem [i-l+st ate*trls . path_lgth] ; 
trls .pmem[state*trls .patb_lgth] = 

stbl . input [trls . prv_stbl_idx [state] ] ; 


> 

for(state=0;state<num_states;state++) { /*** copy to old *♦**/ 

f or (i=0 ; i<trls .path.lgth; i++) 

trls .pmem_old[i+state*trls .path_lgth] = 

trls.pmem[i+state*trls .path.lgth] ; 

» 


void i'nit_fbsig (stable stbl, trellis trls, int M, int L, int nsymM 
int i,j; 
int num_ states; 

num_states=pow( (double) M, (double)L) ; 
f or (i=0; i<M*num_ states ; i++) { 
trls.fbsig[i]=0; 
for(j=0; j<nsym; j++) { 
trls.fbsig[i]+= 

stbl. output [i*nsym+j] . re*stbl. output [i*nsym+j] .re 
+stbl . output [i*nsym+j] . im*stbl . output [i*nsym+j] . im; 

> 

» 


void updt_fb (stable stbl, cmplx.dbl *ch_output, 

trellis trls, int M, int L, int nsym) { 

int i, j ; 

int num_ states; 

num_states=pow( (double)M, (double)L) ; 
for(i=0;i<M*num_states;i++) { 
trls.fb[i]=0; 
for(j=0; j<nsym; j++) { 

trls. fb[i]+=ch_ output [j] . re*stbl. output [i*nsym+j] .re 

+ch_ output [j] . im*stbl . output [i*nsym+j] . im; 

> 

» 


void get_cb_out (stable stbl_tx,int tx_out_idx, 
cmplx_dbl *ch_output ,int nsym) 


int i ; 


i 
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for(i=0;i<nsym;i++) { 

ch_output[i] .re=stbl_tx. output [tx_out_idx*nsym+i] .re; 
ch_ output [i] . im=stbl_tx . output [tx_out_idx*nsym+i] . im ; 

» 

void add.awgn (double esavg, double esnodb, 

cmplx_dbl *ch_output , int nsym ) { 
double ul ,u2,nl ,n2,No,esno; 
int j; 

esno=pow(10.0,esnodb/10.0) ; 

No=esavg/esno ; 
for ( j =0 ; j <nsym ; j ++ ) { 
ul= ( (double ) rand ())/ (RAND.MAX) ; 
u2=( (double) rand ())/(RAND_MAX) ; 
if (ul < le-8) ul=le-8; 

ch_ output [ j] . re+=sqrt (No/2) *sqrt (-2*log(ul) ) *cos (2*3 . 141593*u2) ; 
ch_ output [ j ] . im+^sqrt (No/2) *sqrt (-2*log (ul) ) *sin (2*3 . 141593*u2) ; 
» 
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A.4. Program for Calculating d r 


/********************************************************/ 


/ * dmin2 . c : 

A program for calculating the minimum 

*/ 

/* 

Euclidean distance between channel 

*/ 

/* 

sequences based on a state table 

*/ 

/* 

description of the channel. The state 

*/ 

/* - 

table is assumed to contain the 

*/ 

/* 

oversampled outputs of the channel . 

*/ 

/* 

The program is based on the algorithm 

*/ 

/* 

described in "Digital Transmission 

*/ 

/* 

Theory," by Benedetto, Biglieri, and 

*/ 

/* 

Castellani, Prentice Hall, 1987. 

*/ 

/* 

chapter 10. 

*/ 


/I**#*******************#************:*:********************/ 

#include <stdio.h> 

#include <math.h> 

#include <stdlib.h> 
typedef struct { 
double re; 
double im ; 

}cmplx_dbl ; 
typedef struct { 
float re ; 
float im; 

}cmplx_f It ; 
typedef struct { 


int 

int 

cmplx.flt 
} stable; 


♦input ; 
♦state; 
♦output ; 


int main () 
int M; 
int L; 
int flag; 
int n; 
int i,j; 


/♦symbol set size*/ 
/♦channel memory*/ 

/♦misc flag */ 

/♦path length variable*/ 
/♦misc variables*/ 
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FILE *st_tbl_f ile; 
int num_states ; 
int nsym; 
stable stbl ; 
double *Dn,*Dn_old; 
int *PM; 
double dmin; 

st_tbl_f ilename [100] ; 

load.st _tbl_cmplx (FILE *st_tbl_f ile , 

stable stbl, int M,int L,int nsym); 

print_st_tbl_cmplx (stable stbl, int M.int L,int nsym); 

build_PM (stable stbl, int *PM, int M, int L) ; 
init.Dn (double *Dn,int *PM, stable stbl, int M.int L,int nsym); 
updt_Dn (double *Dn, double *Dn_old,int *PM, 

stable stbl, int M.int L,int nsym); 
stat_Dn (double *Dn,int M,int L,int *f lag, double *dmin) ; 

/****************/ 

/** User Input **/ 

/ ****************/ 
printf ("Enter Symbol Set Size M: "); 
scanf ("’/,d" ,&M) ; 

printf ("Enter Samples per Symbol: "); 
scanf ("*/,d" ,&nsym) ; 
printf ("Enter Channel Memory L: "); 
scanf ("*/,d" ,&L) ; 

printf ( "Enter State Table Filename: "); 
scanf ("’/.s" , st _tbl_f ilename) ; 

if ((st_tbl_file=fopen(st_tbl_f ilename, "r"))==NULL) { 
printf ("ERROR: could not open \"*/.s\" for reading\n" , 
st_tbl_f ilename) ; 
exit(l) ; 

> 

/* memory allocation */ 
num_states=pow((double)M, (double)L) ; 
stbl. input =( int *)malloc(M*num_states*sizeof (int)) ; 
stbl.state=(int *)malloc(M*L*num_states*sizeof (int)) ; 
stbl . output = ( cmplx_f It * ) malloc (nsym*M*num_st ates* 
sizeof (cmplx_f It) ) ; 

PM=(int *)mcLlloc(num_states*M*sizeof (int)) ; 
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char 

void 

vo*id 

void 

void 

void 

void 


/♦state table filename */ 
/♦number of channel states ♦/ 
/♦samples per symbol */ 

/♦state table ♦/ 

/♦matrix of min sqrd distances*/ 
/♦predecessor matrix*/ 

/♦minimum distance squared */ 



Dn=(double *)malloc(num_states*num_states*sizeof (double)) ; 
Dn_old= (double *)malloc(num_states*num_states*sizeof (double) ) ; 

/* load state table */ 

load_st_tbl_cmplx (st_tbl_f ile , stbl , M , L , nsym) ; 


build_PM ( stbl , PM , M , L) ; 
n=0; 

init _Dn (Dn , PM , stbl , M , L , nsym) ; 
flag-1 ; 

while (flag>0) ■[ 

updt _Dn (Dn , Dn_old , PM , stbl , M , L 

stat JDn(Dn,M,L,&f lag,&dmin) ; 
n++; 

print f ( "n= # /,d d='/,g\n" , n , dmin) ; 

> 


/* build Predecessor Matrix */ 

/* step la */ 

/* init seq. length variable */ 
/* Initialize Squared Minimum */ 
/* Distance Matrix "Dn" */ 

/* step lb and step 2 */ 


/* Update Distance Matrix */ 
nsym) ; /* step 3 */ 

/* Check if Done */ 

/* step 4 */ 

/* update seq. length */ 

/* print stats */ 


} 


printf("\nn = %d, 
return (0) ; 


dmin~2 = */.g\n" , 

n,dmin); /* print final stats */ 


void load_st_tbl_cmplx(FILE *st_tbl_f ile, stable stbl, 

int M,int L,int nsym) { 

int num_states; 
int i , j ; 

num_states=pow( (double) M, (double) L) ; 
for(i=0;i< M*num_states ; i++) { 

f-scanf (st_tbl_f ile, "y,d" ,&stbl . input [i] ) ; 

for (j=0; j< L; j++) f scanf (st_tbl_f ile, "yd" ,&stbl . state [j+i*L] ) ; 
f or ( j =0 ; j <nsym ; j ++) 

f scanf (st_tbl_f ile , "*/,f */.f i" , 

festbl . output [i*nsym+ j ] . re , festbl . output [i*nsym+ j ] . im) ; 
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void print_st_tbl_cmp lx (stable stbl, int M,int L, int nsym) { 
int num_states; 
int i,j; 

num_states=pov((double)M, (double)L) ; 
for(i=0;i<M*num_states;i++) { 
printf("%d ", stbl . input [i] ) ; 

for (j=0; j<L; j++) printf("%d ",stbl.state[j+i*L]); 
for (j=0; j<nsym; j++) 

'printfC */.g %gi" , stbl .output [i*nsym+j] .re , 

stbl . output [i*nsym+j] . im) ; 

printf ("\n") ; 

> 


void build_PM (stable stbl, int *PM, int M, int L) { 
int pstate_idx,pstate,state, j ,num_states; 
int get_snum( stable stbl, int stbl_idx,int M, int L) ; 
num_states=pow((double)M, (double)L) ; 

f or(state=0;state<num_states; state++) { 
for (j=0;j<M; j++) { 

pstate_idx=state+j*pow((double)M, (double)L) ; 

/*j-th predecessor of state i*/ 
PM[j+M*state]=get_snum(stbl,pstate_idx, M, L) ; 

» 


/* get state number from state table index */ 
int get_snum(stable stbl, int stbl_idx,int M, int L) -( 
int i ; 
int snum=0 ; 
for (i=0;i<L; i++) 

snum+=stbl . state [i+stbl_idx*L] *pow( (double) M, (double) (i) ) ; 
return (snum) ; 

} 

void init_Dn (double *Dn,int *PM, stable stbl, int M,int L,int nsym){ 
int num_states,i, j ,k; 

int pj,pi; /* previous i and j */ 

int Spi,Spj; /* states pi and pj */ 
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int Spi_stbl_idx; 
int Spj_stbl_idx; 
double d2,dif_re,dif_im; 
double dij_min=-l; 
double dSpii_Spjj; 
int flag; 
int N; 

num_states=pow((double)M, (double)L) ; 

N=num_states; 

f or (i=0; i<num_ states ; i++){ /*step la*/ 

f or(j=0; j<num_states ; j++){ 

Dn [j+i*num_states] =-l ; 

' » 

for(i=0;i<num_states;i++) { /*step 2 */ 

f or(j=i+l ; j<num_states; j++) { 

-flag-1; 

f or(pi=0;pi<M;pi++) { 

Spi=PM [pi+i*M] ; 
for(pj=0;pj<M;pj++) { 

Spj=PM[pj+j*M] ; 
if (Spi==Spj) { 

Spi_stbl_idx=i+pi*N ; 

Spj _stbl_idx=j+p j *N ; 
d2=0; 

f or (k=0;k<nsym;k++) { 

dif _re=stbl . output [Spi_stbl_idx*nsym+k] . re 
-stbl . output [Spj_stbl_idx*nsym+k] . re ; 
dif _im=stbl . output [Spi_stbl_idx*nsym+k] . im 
-stbl. output [Spj _stbl_idx*nsym+k] .im; 
d2+=dif_re*dif_re+dif_im*dif_im; 

> 

dSpii_Spj j=d2; 

if Cflag==l) {dij_min=dSpii_Spjj ;flag=0;>; 
if (dSpii_Spj j<dij_min) dij_min=dSpii_Spj j ; 

> 

> > 

if (f lag==0) { 

Dn [ j +i*num_states] =di j _min ; 

Dn [i+ j *nvim_states] =di j _min ; 

- > 

> > 

> 

J* Update Squared Minimum Distance Matrix, step 3 */ 
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void updt_Dn (double *Dn, double *Dn_old,int *PM, stable stbl, 
int M.int L,int nsym) { 
int num_st at es , i , j , k ; 
double pi ,pj ; 
int Spi , Sp j ; 

double dSpii_Spj j ,dij_min; 
double d2,dif_re,dif_im; 
int Spi_stbl_idx; 
int Spj_stbl_idx; 
int N, flag; 

N=pov((double)M, (double)L) ; 
for (i=0;i<N;i++) { 
for(j=0; j<N; j++) { 

Dn_old[j+i*N]=Dn[j+i*N] ; 

» 

for(i=0;i<N;i++){ 
for(j=i; j<N; j++){ 
f lag=l ; 

f or (pi=0 ; pi<M ; pi++) { 

Spi=PM[pi+i*M ] ; 
f or (p j =0 ; pj <M ; p j ++) { 

Spj=PM[pj+j*M] ; 
if (Dn_old[Spj+Spi*N]>0) { 

Spi_stbl_idx=i+pi*N ; 

Spj _stbl_idx=j+pj *N ; 
d2=0; 

for (k=0 ; k<nsym ; k++) { 

dif_re=stbl. output [Spi_stbl_idx*nsym+k] .re 
-stbl . output [Spj _stbl_idx*nsym+k] . re ; 
dif _im=stbl . output [Spi_stbl_idx*nsym+k] . im 
-stbl . output [Spj _stbl_idx*nsym+k] . im ; 
d2+=dif_re*dif _re+dif_im*dif _im; 

> 

dSpii.Spi j=d2; 
if (flag==l ) { 

if (Dn_old[Spj+Spi*N]==-l) dij_min=dSpii_Spj j ; 
else dij_min=Dn_old[Spj+Spi*N]+dSpii_Spjj ;flag=0; 
> 

if (Dn_old[Spj+Spi*N]==-l) { 

if( dSpii_Spjj <= dij.min ) dijlmin=dSpii_Spj j ; 
else 

if ((Dn_old[Spj+Spi*N]+dSpii_Spj j) <= dij_min) 
di j _min=Dn_old [Spj +Spi*N] +dSpii_Sp j j ; 

> 


/♦predecessor i and j*/ 
/* State pi and pj */ 
/*sqrd distance*/ 


/* number of states */ 
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> 

» 

if (i==j && flag==0) { 

if (Dn[j+i*N]==-l) Dn[j+i*N]=dij_min; 

else if (dij_min<Dn[j+i*N] ) Dn[j+i*N]=dij_min; 

> 

if(flag==0 && j>i ) { 

Dn [ j +i*N] =di j _min ; 

Dn [i+ j *N] =di j _min ; 

> 

» 

> 

- /* Status of Dn, step 4 */ 

void stat.Dn (double *Dn,int M,int L,int *f lag, double *dmin) { 
int i, j ,num_states; 
double dmin_tmp; 
double flag_tmp=0; 
num_states=pow( (double) M, (double)L) ; 
dmin_tmp=Dn [0] ; 
for (i=l;i<num_states;i++) { 

if (Dn[i+i*num_states]< dmin_tmp) dmin_tmp=Dn[i+i*num_states] ; 

> 

*dmin=dmin_tmp ; 
if (dmin_tmp>0) { 

for (i=0;i<num_states;i++) { 
for (j=0; j<num_states; j++) { 
if (Dn[i, j]<dmin_tmp) flag_tmp=l; 

»} 

else flag_tmp=l; 

* f 1 ag=f 1 ag_ t mp ; 

} 


137 



